RAG Pipelines in Laravel: Chunking, Embedding, and Retrieval with pgvector

RAG Pipelines in Laravel Without the Magic Box

Retrieval-Augmented Generation (RAG) is the backbone of most production AI features: you embed your own documents, store the vectors, and retrieve the most relevant chunks before sending them to an LLM. Laravel has everything you need to build this cleanly — no Python microservice required.

This article focuses on the ingestion side: chunking, embedding, and storing. Retrieval and prompt assembly follow naturally once the foundation is solid.

1. Schema: Storing Vectors in PostgreSQL

Install the pgvector extension and add a vector column to your documents table.

CREATE EXTENSION IF NOT EXISTS vector;

// database/migrations/xxxx_create_document_chunks_table.php
Schema::create('document_chunks', function (Blueprint $table) {
    $table->id();
    $table->foreignId('document_id')->constrained()->cascadeOnDelete();
    $table->text('content');
    $table->integer('chunk_index');
    $table->vector('embedding', 1536); // text-embedding-3-small dimensions
    $table->timestamps();
});

// Add an HNSW index for fast approximate nearest-neighbour search
DB::statement('CREATE INDEX ON document_chunks USING hnsw (embedding vector_cosine_ops)');

The vector column type requires the pgvector/pgvector-php package or raw DB statements. Keep dimensions consistent with your chosen model.

2. Chunking Strategy

Naive line-splitting loses context at boundaries. A sliding-window approach with overlap preserves sentence continuity.

final class TextChunker
{
    public function __construct(
        private readonly int $maxTokens = 400,
        private readonly int $overlapTokens = 50,
    ) {}

    /** @return list<string> */
    public function chunk(string $text): array
    {
        // Rough token estimate: 1 token ≈ 4 characters for English
        $chunkSize = $this->maxTokens * 4;
        $overlap = $this->overlapTokens * 4;
        $chunks = [];
        $offset = 0;
        $length = strlen($text);

        while ($offset < $length) {
            $slice = substr($text, $offset, $chunkSize);

            // Break at the last sentence boundary within the slice
            if ($offset + $chunkSize < $length) {
                $boundary = strrpos($slice, '. ');
                if ($boundary !== false) {
                    $slice = substr($slice, 0, $boundary + 1);
                }
            }

            $chunks[] = trim($slice);
            $offset += strlen($slice) - $overlap;
        }

        return array_filter($chunks);
    }
}

Character-based estimation is imprecise but avoids a tokeniser dependency. For stricter control, use tiktoken-php.

3. Embedding via a Queued Job

Embedding is I/O-bound and rate-limited — always do it asynchronously.

final class EmbedDocumentChunks implements ShouldQueue
{
    use Dispatchable, Queueable;

    public int $tries = 3;
    public int $backoff = 10;

    public function __construct(private readonly int $documentId) {}

    public function handle(TextChunker $chunker, OpenAIClient $openai): void
    {
        $document = Document::findOrFail($this->documentId);
        $chunks = $chunker->chunk($document->body);

        // Batch embed: OpenAI accepts up to 2048 inputs per request
        $response = $openai->embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $chunks,
        ]);

        $rows = [];
        foreach ($response->embeddings as $i => $embedding) {
            $rows[] = [
                'document_id' => $document->id,
                'chunk_index' => $i,
                'content' => $chunks[$i],
                // Cast float[] to pgvector literal
                'embedding' => '[' . implode(',', $embedding->embedding) . ']',
                'created_at' => now(),
                'updated_at' => now(),
            ];
        }

        // Delete stale chunks before re-inserting
        DocumentChunk::where('document_id', $document->id)->delete();
        DocumentChunk::insert($rows);
    }
}

4. Retrieval: Cosine Similarity Query

final class ChunkRetriever
{
    public function __construct(private readonly OpenAIClient $openai) {}

    /** @return Collection<int, DocumentChunk> */
    public function retrieve(string $query, int $limit = 5): Collection
    {
        $vector = $this->embed($query);

        return DocumentChunk::query()
            ->orderByRaw('embedding <=> ?', [$vector])
            ->limit($limit)
            ->get();
    }

    private function embed(string $text): string
    {
        $response = $this->openai->embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $text,
        ]);

        return '[' . implode(',', $response->embeddings[0]->embedding) . ']';
    }
}

The <=> operator is pgvector's cosine distance. Swap for <#> (inner product) or <-> (L2) depending on your model's recommendation.

Key Takeaways

• Chunk with overlap to avoid losing context at boundaries; sentence-boundary snapping improves coherence.
• Batch embed in a single API call per document to stay within rate limits and reduce latency.
• Delete-then-insert on re-ingestion keeps chunks consistent without complex upsert logic.
• HNSW indexes on the embedding column are essential for sub-millisecond retrieval at scale.
• Keep the chunker, embedder, and retriever as small, injectable classes — they're easy to unit-test and swap.