Embeddings and vectors

Two bindings together make retrieval work. env.ai.embed turns text into a vector, and env.vectors keeps vectors and finds the nearest ones to a query. Both come with every project. The vectors are rows in your project's own database, which is created the first time your app uses it.

Embedding text

const r = await env.ai.embed("@runlot/titan-embed-v2", [
  "Refunds are issued within 5 business days.",
  "Shipping is free over $50.",
]);

r.vectors     // number[][] — one per input, in order
r.dimensions  // 1024
r.usage       // { inputTokens }

input is one string or an array of strings. There is no output side to an embedding, so usage carries input tokens only and the price is per million input tokens.

The embedding models sit in the same catalog as the chat models, with kind: "embedding".

The value model runs on the free daily allowance like every @runlot/… model. Embeddings share the per-minute limit with env.ai.run — it is the same upstream budget — and a call counts once per upstream call, not once in total: these models take one text per request, so sixty texts is sixty of the minute. They do not share env.ai.run's concurrency limit, which bounds how much an unpaid call can commit; an embedding commits a fraction of a cent, so what bounds it is the minute.

An embedding is only useful against vectors made by the same model, at the same width, under the same taskType. Pick one for a namespace and keep it. The same text under two task types is a different vector, so fill a namespace and query it under the same value, and write down which one you used — that is why the answer echoes taskType back. Changing it later means re-embedding everything you stored.

There are two reasons to reach for it. For search there is the RETRIEVAL_QUERY / RETRIEVAL_DOCUMENT pair, which projects a question and a passage differently. For everything that is not search there is no substitute at all — CLASSIFICATION and CLUSTERING ask for a space that is not for nearest-neighbour search, which is what you want when the vectors go into a classifier or a clusterer. @runlot/titan-embed-v2 has no such notion and refuses a value.

The vector store

await env.vectors.upsert([
  { id: "faq-1", values: r.vectors[0], metadata: { lang: "en", topic: "refunds" } },
  { id: "faq-2", values: r.vectors[1], metadata: { lang: "en", topic: "shipping" } },
]);

These are the calls env.vectors takes.

An item is { id, values, namespace?, metadata? }. values is an array of numbers or a Float32Array; metadata is a small JSON object (10 KiB at most); namespace defaults to "default" and matches ^[a-z0-9_-]{1,64}$. One call takes up to 1,000 items.

Querying

const q = await env.ai.embed("@runlot/titan-embed-v2", "how long do refunds take?");
const hits = await env.vectors.query(q.vectors[0], {
  topK: 3,
  filter: { lang: "en" },
  returnMetadata: true,
});
// hits.matches → [{ id: "faq-1", score: 0.91, metadata: {…} }, …]

The filter runs before the scoring, so topK means the top K of the rows that match — not the matches among the top K overall.

For cosine and dot a higher score is nearer; for euclidean the score is a distance and lower is nearer. Cosine is computed on normalised vectors, so it is the right default for text embeddings.

A query whose width does not match the namespace's is refused with dimension_mismatch. A namespace holds one width; mixing them is refused at query time with mixed_dimensions.

A RAG endpoint, end to end

export default {
  async fetch(request, env) {
    const url = new URL(request.url);

    // Index: POST /index with { id, text }
    if (request.method === "POST" && url.pathname === "/index") {
      const { id, text } = await request.json();
      const e = await env.ai.embed("@runlot/titan-embed-v2", text);
      await env.vectors.upsert([{ id, values: e.vectors[0], metadata: { text } }]);
      return Response.json({ ok: true });
    }

    // Ask: GET /ask?q=…
    const question = url.searchParams.get("q") ?? "";
    const e = await env.ai.embed("@runlot/titan-embed-v2", question);
    const hits = await env.vectors.query(e.vectors[0], { topK: 3, returnMetadata: true });
    const context = hits.matches.map((m) => m.metadata.text).join("\n\n");

    const answer = await env.ai.run("@runlot/nova-micro", {
      system: "Answer from the context. If it is not there, say so.",
      prompt: `Context:\n${context}\n\nQuestion: ${question}`,
    });
    return Response.json({ answer: answer.text, sources: hits.matches.map((m) => m.id) });
  },
};

How the search works, and what it costs

Search is exact, not approximate. There is no ANN index. A query computes the distance to every vector in the namespace and returns the nearest — the answer is always the true nearest neighbours, and the cost grows linearly with the namespace. At the plan's bounds that is a few milliseconds; the arithmetic is the cheap part.

The first query after a wake pays a load. The vectors live in your database as bytea. The worker reads a namespace into memory on its first query, keeps it warm while the process lives, and updates it on every write it makes. A project that was parked comes back cold, so the first query after a wake reads the namespace once. A write that bypasses the binding — a DELETE from psql — is noticed on the next query and the namespace is reloaded.

The schema is yours. runlot_vector.items is a table in your project's database: it counts toward your database size, it shows up in psql, and it is restored with your backups. Do not edit the vec column by hand; everything else is fair to read.

Limits

The vector cap is a memory bound, not a quota: the warm index shares your worker's memory cap with your code and the database engine, and 25,000 vectors of 3,072 dimensions is about 300 MB of it. A write that would cross the cap is refused whole with vector_limit — nothing in that call lands. A namespace that grew past the cap through psql is refused at query time with namespace_too_large rather than searched partially.

Errors

env.vectors throws a VectorsError with e.code:

env.ai.embed throws like env.ai.run (e.status, e.code), plus wrong_kind when a chat model is passed to embed or an embedding model to run, and bad_dimensions when the width asked for is not one the model answers.

CLI

runlot vectors                # namespaces: count, dimensions, and the plan's bound
runlot vectors describe <ns>  # one namespace
runlot vectors delete <ns>    # delete every vector in the namespace (-y)

The dashboard's AI tab shows the same numbers. There is no command that writes or queries a vector — env.vectors is reachable only inside the worker, the same boundary as env.ai.

What is not here

• An approximate index. Search is exact, so query time grows with the size of the namespace, and the per-plan caps are set where that stays fast. If your collection is larger than a cap, split it across namespaces or across projects.
• Range or inequality filters. The filter is equality on top-level keys.
• SQL over vectors. runlot_vector.items is a plain table; there is no <-> operator and no join on similarity. Similarity search goes through env.vectors.
• Local runlot dev. Both work locally once you are logged in: env.ai.embed is relayed through the control plane and billed to the org's AI credit as lane = dev, and env.vectors lives in the local database (exact k-NN, as deployed). RUNLOT_DEV_AI=off refuses env.ai for a run.