libSQL Embeddings: SQLite Vector Search for RAG

Introduction: libSQL embeddings for semantic search without another database

libSQL embeddings store semantic vectors beside SQLite-style data, enabling fast semantic search, recommendations, and retrieval-augmented generation without a separate vector database. They work especially well for phone, edge, and small serverless applications.

Turso is built on libSQL and supports modern application database workflows..

libSQL stores and searches embeddings but does not create them. An embedding model converts text or images into numeric arrays; libSQL supplies the SQLite vector storage, distance functions, and indexes to search them.

Covered topics:

• How native vector storage works
• Exact and indexed Turso vector search
• Hybrid vector SQL queries for a lightweight RAG database
• Practical limits, deployment choices, and common mistakes

What native libSQL embeddings provide for semantic search

libSQL is Turso’s open-source SQLite fork. It preserves the SQLite file format and API while adding remote access, embedded replicas, and native vector operations. Stock SQLite lacks these libSQL vector functions.

libSQL encodes vectors as BLOBs with type and dimension metadata rather than adding a new SQLite storage class. Application fields and embeddings can therefore share a row and transaction.

Part	What it does	Where it runs
Embedding model	Converts content into a numeric vector	Model API or local model
Vector column	Stores the numeric representation	libSQL database
Distance function	Measures how closely two vectors match	SQL query
Vector index	Finds approximate nearest neighbors faster	libSQL database
Application	Applies permissions, filters, and RAG logic	Server, edge, or device

Changing an embedding model usually requires regenerating vectors, while titles and permissions remain ordinary SQL updates. Storing both together also reduces the risk of vector data drifting from the main database.

How SQLite vector similarity powers semantic search

An embedding is an ordered list of numbers.

Similar meanings tend to produce nearby vectors under the chosen distance measure. Compared vectors must have the same type and dimensions.

The libSQL AI and embeddings documentation recommends starting with FLOAT32 or F32_BLOB. A 1,536-dimensional F32_BLOB uses 6,144 bytes for raw values, or about 586 MiB across 100,000 rows before table and index overhead.

Type	Approximate storage	Practical use
F32_BLOB(D)	4D bytes	Normal starting point for dense embeddings
F16_BLOB(D)	2D + 1 bytes	Smaller storage after recall testing
F8_BLOB(D)	D + 14 bytes	Roughly 4 times smaller than float32 at high dimensions
F1BIT_BLOB(D)	ceil(D / 8) + 3 bytes	Very compact models designed for binary vectors

Text usually uses cosine distance: vector_distance_cos ranges from approximately 0 to 2, where lower values mean closer matches. vector_distance_l2 measures Euclidean distance. L2 is unavailable for one-bit vectors. libSQL limits vectors to 65,536 dimensions, far above most current text embeddings.

Build your first libSQL embeddings table for a RAG database

Store enough information to reproduce and replace each embedding. Recording the model name prevents comparisons between vectors from different models.

1. Choose one embedding model and confirm its output dimension.

2. Split long documents into focused chunks. Start experiments at 300 to 800 tokens per chunk with 10% to 20% overlap, then adjust using real searches.

3. Store the text, ownership fields, and model identifier with the vector.

4. Generate embeddings outside libSQL and insert them with vector32; applications can bind the JSON array as a prepared-statement parameter.

CREATE TABLE knowledge_chunks (
  id INTEGER PRIMARY KEY,
  source_id INTEGER NOT NULL,
  tenant_id TEXT NOT NULL,
  locale TEXT NOT NULL,
  title TEXT NOT NULL,
  body TEXT NOT NULL,
  embedding_model TEXT NOT NULL,
  embedding F32_BLOB(1536) NOT NULL,
  created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX knowledge_tenant_idx
ON knowledge_chunks(tenant_id, locale);

INSERT INTO knowledge_chunks (
  source_id, tenant_id, locale, title, body,
  embedding_model, embedding
) VALUES (?, ?, ?, ?, ?, ?, vector32(?));

The final parameter must contain exactly 1,536 numbers. Reject missing, malformed, or incorrectly sized arrays before insertion. Because libSQL stores embeddings in binary, retaining the original JSON usually wastes space unless needed for debugging.

Exact SQLite vector search and Turso vector search indexes

An exact similarity query scans eligible rows, calculates each distance, and sorts the results:

SELECT id, title,
       vector_distance_cos(embedding, vector32(?)) AS distance
FROM knowledge_chunks
WHERE tenant_id = ? AND locale = ?
ORDER BY distance ASC
LIMIT 5;

A conventional SQL index can narrow tenant and locale before calculating distances and may suffice for a small collection. No universal row count requires indexing; dimensions, hardware, query rate, and filter selectivity determine when it helps.

For larger collections, create a native DiskANN index and query it explicitly with vector_top_k:

CREATE INDEX knowledge_embedding_idx
ON knowledge_chunks(
  libsql_vector_idx(embedding, 'metric=cosine')
);

WITH candidates AS (
  SELECT id
  FROM vector_top_k(
    'knowledge_embedding_idx', vector32(?), 50
  )
)
SELECT k.id, k.title,
       vector_distance_cos(k.embedding, vector32(?)) AS distance
FROM candidates AS c
JOIN knowledge_chunks AS k ON k.id = c.id
ORDER BY distance ASC
LIMIT 5;

Approach	Result quality	Main cost	Best starting use
Exact scan	Exact nearest rows	Distance calculation for every eligible row	Small or heavily filtered datasets
vector_top_k	Approximate neighbors	Extra index storage and maintenance	Larger datasets or frequent searches

The index tracks table changes, but search is approximate. Start with the exact query, measure it, and add the index when latency or throughput requires it.

Write hybrid vector SQL queries for a RAG database

Vector search alone cannot ensure reliable RAG: a semantically close chunk may belong to the wrong customer, language, product version, or publication state. Hybrid vector SQL uses semantic similarity for discovery and SQL for factual restrictions.

A lightweight RAG request can follow:

1. Convert the user’s question with the ingestion embedding model.
2. Retrieve more candidates than needed.
3. Join those candidates to relational tables and enforce access rules.
4. Recalculate exact distances and keep the best permitted chunks.
5. Send selected text and source URLs to the language model.

WITH candidates AS (
  SELECT id
  FROM vector_top_k(
    'knowledge_embedding_idx', vector32(?), 60
  )
)
SELECT k.id, k.title, k.body, s.url,
       vector_distance_cos(k.embedding, vector32(?)) AS distance
FROM candidates AS c
JOIN knowledge_chunks AS k ON k.id = c.id
JOIN sources AS s ON s.id = k.source_id
WHERE k.tenant_id = ?
  AND k.locale = ?
  AND s.status = 'published'
ORDER BY distance ASC
LIMIT 6;

The query fetches 60 candidates to return 6 after filtering. A 10:1 ratio is an experiment, not a guarantee. If filters remove most candidates, enlarge the pool, create a partial vector index for a stable condition, or isolate each tenant in its own database.

For exact product names or error codes, combine vector candidates with SQLite FTS5 results and rerank them in application code. Semantic and lexical matching serve different needs.

libSQL as an edge database for mobile, serverless, and marketing data

The strongest reason to use SQLite vectors at the edge is architectural restraint. If an application already uses libSQL, another service may create more synchronization work than the search warrants.

Example	Embedded content	Useful SQL filter	Result
Campaign knowledge search	Briefs, approved claims, and reports	Brand, locale, channel, publication date	Finds reusable work without crossing brand rules
Support assistant	Manuals, release notes, and resolved tickets	Product version and customer entitlement	Produces a grounded answer draft with source links
Product recommendations	Descriptions and product attributes	In-stock status, category, region, and price	Returns similar products that can actually be purchased
Personal mobile assistant	Notes and conversation memories	Date, person, or privacy class	Retrieves context locally for a private response

The Kin personal assistant case study describes keeping user data and vector retrieval on the device. Though vendor-produced rather than independently benchmarked, it shows the appeal of one transactional database on a resource-constrained phone.

A remote libSQL database spares serverless deployments from maintaining a database server. At the edge, local files or replicas can reduce read latency when persistent storage is available. Check the SDK and its write behavior. Some edge platforms lack durable local filesystems, while older libSQL embedded replicas send writes to a remote primary.

libSQL versus other vector storage options

Many guides blur that libSQL and the newer Turso Database engine are related but separate projects. Turso’s current TypeScript SDK documentation calls @libsql/client a supported, battle-tested option for existing libSQL applications but recommends newer Turso packages for many new local and synchronization projects.

Option	Strong fit	Trade-off
libSQL native vectors	Existing SQLite-style application needing SQL, vectors, and a small operational footprint	Inherits SQLite’s single-writer foundation and requires a libSQL runtime
New Turso Database engine	New Turso projects needing current local-first and synchronization work	Separate engine and SDK; verify that required vector-index features are available in the chosen release
Stock SQLite plus extension	Local application that can package and load an extension	Extension deployment may be blocked or inconsistent across hosts
PostgreSQL with pgvector	Existing PostgreSQL system with concurrent writes and mature operations	Requires a database server and more infrastructure than an embedded file
Dedicated vector database	Very large distributed collections or specialized vector operations	Another service, bill, permission system, and synchronization path

A normal SQLite client can read vector BLOBs but cannot run vector_distance_cos, libsql_vector_idx, or vector_top_k. Test with the engine and SDK used in production. Because Turso vector search can refer to capabilities across related Turso products, check engine-specific documentation instead of assuming examples are interchangeable.

Common libSQL embedding pitfalls and checks

Weak semantic search usually fails from data preparation or filtering, not mysterious cosine distance. A small evaluation set catches these problems early.

Item	What to check	Why it matters
Model consistency	Indexed rows and queries share a model version and dimensions	Mixed vector spaces produce meaningless distances
Chunk quality	Chunks contain enough context without mixing unrelated subjects	Poor chunks give the model incomplete or noisy evidence
Permission filters	SQL enforces tenant and access conditions after candidate retrieval	Similarity is not an authorization mechanism
ANN recall	Compare indexed and exact results for 50 to 100 representative questions	Approximate search can miss the true nearest rows
Storage growth	Count raw vectors, index size, and duplicate text	High-dimensional libSQL embeddings can dominate the database file
Model migration	Write new vectors to a separate column or table before cutover	In-place model replacement temporarily mixes results

Short answers to common questions:

Conclusion: start small with libSQL vector search for semantic search

libSQL stores, updates, filters, and searches embeddings as ordinary SQL data. This suits lightweight RAG, mobile applications, edge deployments, and modest serverless systems where a separate vector service adds more work than value.

Start small:

1. Embed a representative set of documents with one recorded model.
2. Test an exact SQLite vector query with real user questions and access filters.
3. Add a Turso vector search index only after measuring speed and recall.

Database choice is only one factor. Clean chunks, consistent embeddings, strict permissions, and honest evaluation affect answer quality more than another infrastructure layer.