Building RAG in Laravel: Four Ingestion Bugs That Silently Wreck Retrieval

· 8 min read
laravelairagpgvectorlaravel-ai-sdk
Building RAG in Laravel — a document sliced into stacked chunk strips by glowing emerald cut lines, with one chunk torn clean in half by a jagged red fracture, illustrating an ingestion pipeline that silently breaks retrieval

You followed the build tutorial. Chunk the docs, embed them, store the vectors in pgvector, point an agent at it. The demo works. Your support agent answers questions from your knowledge base.

Then retrieval starts missing. A customer asks about shipping and gets your warranty policy. Or you shrink your embedding dimensions to save storage, redeploy, and half the answers come from the wrong document.

No error. No exception. No failed test.

The build tutorial never warned you, because the build tutorial never checked.

The build guides stop at “it answers”

Every Laravel RAG guide ends in the same place. Enable pgvector, chunk, embed, store, retrieve, answer. The agent replies on screen, the post wraps up. None of them ask whether retrieval is any good, because none of them measure it.

So the choices you copy-paste become load-bearing without anyone telling you.

Ingestion is where retrieval quality is set, before the model runs once. How you chunk, which index operator you pick, how many dimensions you store, which embedding model you use. Each one has a silent failure mode. Get any of them wrong and nothing crashes. Retrieval just quietly gets worse, and the agent stays fluent on top of bad context.

This is the prequel to evaluating RAG in Laravel. That post measured retrieval and watched a config change tank it. This one is upstream: the four ingestion decisions the eval grades, each shown as a real bug.

The system is the same support agent as the rest of this series, the one I tested and hardened against prompt injection. Nine policy documents (returns, shipping, refunds, warranty, account, exchanges, damaged items, promotions, privacy), chunked, embedded with text-embedding-3-small, stored in Postgres. The whole project is on GitHub at MujahidAbbas/laravel-rag-evals. All four bugs live in its ingestion path.

Bug 1: your chunk boundary cuts the answer in half

The laziest chunker splits every N characters:

$chunks = str_split($content, 600);

That cuts wherever the 600th byte lands. Mid-sentence. Mid-number.

Your refund policy says refunds appear in “5 to 10 business days,” and the split drops between “5 to” and “10 business days.” Now no single chunk holds the full answer. Retrieval can fetch the right chunk and still hand the model half a fact.

Chunk on meaning instead. Break on paragraph boundaries so a chunk never severs a sentence:

// app/Support/Chunker.php
class Chunker
{
    public function chunk(string $text, int $maxChars = 600): array
    {
        $paragraphs = preg_split('/\n\s*\n/', trim($text));
        $chunks = [];
        $current = '';

        foreach ($paragraphs as $paragraph) {
            $paragraph = trim($paragraph);

            if ($paragraph === '') {
                continue;
            }

            if ($current !== '' && mb_strlen($current) + mb_strlen($paragraph) + 2 > $maxChars) {
                $chunks[] = $current;
                $current = '';
            }

            $current = $current === '' ? $paragraph : $current."\n\n".$paragraph;
        }

        if ($current !== '') {
            $chunks[] = $current;
        }

        return $chunks;
    }
}

How do you know it matters? You measure it.

The eval scores two numbers. source hit@1 asks whether the top chunk came from the right document. answer@1 asks whether that chunk actually held the answer sentence. Even with clean chunking the gap shows: source hit@1 94%, answer@1 84%. Right document, wrong passage, in five of thirty-two questions. Split mid-sentence and that gap widens until the model is articulate and wrong.

Chunk size is the first knob you tune against the eval, not a number you copy from a blog post.

Bug 2: your index and your query disagree, and Postgres says nothing

pgvector has three distance operators. Cosine <=>, L2 <->, inner product <#>. An HNSW index is built for exactly one of them.

The pgvector PHP package’s own README builds the index like this:

CREATE INDEX ON items USING hnsw (embedding vector_l2_ops);

vector_l2_ops indexes the L2 operator <->. But retrieval orders by cosine:

// app/Support/KnowledgeBaseRetriever.php
return collect(DB::select(
    'select id, source, chunk_index, content,
            1 - (embedding <=> ?::vector) as similarity
     from document_chunks
     order by embedding <=> ?::vector
     limit ?',
    [$vector, $vector, $k],
));

Cosine <=> against an L2 index can’t use the index. Postgres drops to a sequential scan over every row. On your 200-chunk demo that’s instant, so you never see it. At two million chunks it’s a timeout.

Match the operator class to the operator you query with:

// database/migrations/..._create_document_chunks_table.php
DB::statement('CREATE EXTENSION IF NOT EXISTS vector');

Schema::create('document_chunks', function (Blueprint $table) {
    $table->id();
    $table->string('source');
    $table->unsignedInteger('chunk_index');
    $table->text('content');
    $table->timestamps();
});

$dimensions = (int) config('knowledge.embedding_dimensions', 1536);
DB::statement("ALTER TABLE document_chunks ADD COLUMN embedding vector({$dimensions})");

DB::statement('CREATE INDEX document_chunks_embedding_idx
    ON document_chunks USING hnsw (embedding vector_cosine_ops)');

Don’t trust that it lines up. Prove it with EXPLAIN. One catch first: on a small table Postgres ignores the index anyway, because scanning a couple dozen rows is cheaper than walking the graph. So you disable sequential scans, which forces the planner to show whether it can use the index at all:

SET enable_seqscan = off;

EXPLAIN ANALYZE
SELECT id, source FROM document_chunks
ORDER BY embedding <=> '[0.02, 0.03, ...]'::vector
LIMIT 4;

With the cosine index, the cosine query rides it:

Limit
  ->  Index Scan using document_chunks_embedding_idx on document_chunks

Rebuild that index with vector_l2_ops and run the exact same query. The L2 index can’t answer a cosine operator, so even with sequential scans turned off Postgres has nowhere to go:

Limit
  ->  Sort
        ->  Seq Scan on document_chunks

That fallback is the bug. Twenty-two rows in the demo, so you never feel it. Every row on every query in production.

This bug got easier to hit. Laravel 13 ships a native whereVectorSimilarTo():

DocumentChunk::query()
    ->whereVectorSimilarTo('embedding', $question)
    ->limit(4)
    ->get();

It auto-embeds the string, orders by distance, returns models. Clean. But it hardcodes the cosine operator <=>, and you can’t change it. Build your index with vector_l2_ops and the prettiest query in your codebase silently scans the whole table. The helper also applies a default 0.6 similarity cutoff, so anything below it disappears with no signal. The convenience hides the exact decision this bug is about.

Bug 3: shrinking dimensions is a migration, not a config flip

pgvector storage grows with the dimension count. text-embedding-3-small gives you 1536 by default, and the model lets you ask for fewer. So someone reads that six months from now, drops it to 16 to cut the bill, and ships.

The config flip hides what actually has to happen. The dimension is baked into the column type:

DB::statement("ALTER TABLE document_chunks ADD COLUMN embedding vector({$dimensions})");

vector(1536) and vector(16) are different column types. Changing the number means a migration plus re-embedding every chunk. Skip the re-embed and your old 1536-dim vectors don’t even fit the column.

And once you re-embed at 16 dimensions, retrieval falls off a cliff. The eval caught it: source hit@1 dropped from 94% to 47%, fifteen of thirty-two questions wrong. “Do you ship to Canada?” started returning the damaged-items policy. “How do I reset my password?” returned the warranty doc. Nothing in the stack complained, and half the answers came from the wrong document.

One more dimension trap. Only the text-embedding-3-* models accept a custom count. ada-002 is fixed at 1536 and rejects the dimensions param outright. That’s why the embedder guards it:

// app/Support/Embedder.php
$pending = Embeddings::for($texts);

// Only the text-embedding-3-* family accepts a custom dimension count.
if (str_starts_with($this->model(), 'text-embedding-3')) {
    $pending->dimensions($this->dimensions());
}

return $pending->generate(model: $this->model())->embeddings;

Your dimension count is a retrieval setting you prove with the eval, not storage you trim.

Bug 4: ingest with one model, query with another, and every distance is noise

This is the quietest one.

Embeddings from different models live in different vector spaces. A 1536-dim vector from text-embedding-3-small and a 1536-dim vector from ada-002 are the same shape and completely incomparable. Compute a distance between them and you get a number that means nothing.

So the model and the dimensions have to match between ingestion and querying. Every time. The way to guarantee that is one source of truth both paths read:

// config/knowledge.php
return [
    // The OpenAI embedding model used for BOTH ingestion and querying.
    // These MUST match: embedding a query with a different model or
    // dimension than the stored chunks produces meaningless distances.
    'embedding_model' => env('KB_EMBEDDING_MODEL', 'text-embedding-3-small'),
    'embedding_dimensions' => (int) env('KB_EMBEDDING_DIMENSIONS', 1536),
    'retrieval_k' => (int) env('KB_RETRIEVAL_K', 4),
];

Ingestion reads the model from config. Retrieval reads the model from config. They can’t drift, because there’s one value behind both. The same Embedder::many() from bug 3 is the only place either path embeds text, which is what makes the rule hold instead of being a comment you hope someone reads.

Skip this and the failure is the worst kind. No error. Plausible-looking results. Slowly, confidently wrong.

The re-ingest tax

The ingest command rebuilds from scratch. Truncate the table, re-embed every chunk, insert:

// app/Console/Commands/IngestKnowledge.php
DB::table('document_chunks')->truncate();

foreach ($files as $file) {
    $source = basename($file);
    $chunks = $chunker->chunk(file_get_contents($file), $maxChars);

    // One embedding request for every chunk in the file.
    $embeddings = $embedder->many($chunks);

    foreach ($chunks as $index => $content) {
        DB::insert(
            'insert into document_chunks (source, chunk_index, content, embedding, created_at, updated_at)
             values (?, ?, ?, ?::vector, now(), now())',
            [$source, $index, $content, $embedder->toVector($embeddings[$index])],
        );
    }
}

Fine for nine documents. At a few thousand, every re-ingest re-embeds text that never changed, and you pay OpenAI for all of it.

The SDK has a fix the demo doesn’t use yet. cache():

Embeddings::for($texts)->dimensions(1536)->cache()->generate(model: $model);

It keys the response on the model, the dimensions, and the exact input texts, for thirty days by default. Re-ingest a file whose chunks didn’t change and the vectors come back from the cache instead of a fresh API call. You pay once for text you embed once.

What this gives you

Before today you copied your ingestion settings and crossed your fingers. Chunk size from a blog post. An index operator from a Stack Overflow answer. A dimension count that “seemed fine.” You found out it was wrong when a customer asked about Canada and heard about damaged goods.

Now you treat each one as a decision you can prove. Tune the chunk size, run the eval, keep the number that scores higher. Match the index ops class to your query operator and confirm it with EXPLAIN. Change embedding models on purpose, knowing it’s a migration and a re-embed, not a toggle. Lock the model behind one config value so ingestion and retrieval can never disagree.

The build guides get you an agent that answers. This gets you an agent that answers from the right page, and a way to know it still does after your next change.

Build it. Prove it. Then tune it.

The whole pipeline is on GitHub at MujahidAbbas/laravel-rag-evals. Clone it, run the eval, and change the chunk size to watch retrieval move.

All articles