RAG in Laravel: pgvector, Embeddings, and Retrieval-Augmented Generation in Practice

Why RAG Belongs in Your Laravel App

Retrieval-Augmented Generation (RAG) is the pattern that lets an LLM answer questions grounded in your data rather than its training corpus. Most tutorials reach for Python and LangChain. You don't need either. Laravel, PostgreSQL with the pgvector extension, and a thin HTTP client to an embeddings API are enough to ship a production RAG pipeline that your team can actually maintain.

This article walks through the full loop: chunking documents, storing embeddings, querying by cosine similarity, and injecting retrieved context into a prompt.

1. Enable pgvector and Create the Schema

Install the extension once on your PostgreSQL instance, then write a migration:

CREATE EXTENSION IF NOT EXISTS vector;

// database/migrations/xxxx_create_document_chunks_table.php
Schema::create('document_chunks', function (Blueprint $table) {
    $table->id();
    $table->foreignId('document_id')->constrained()->cascadeOnDelete();
    $table->text('content');
    $table->integer('token_count');
    // 1536 dims for text-embedding-3-small
    $table->string('embedding', 65535)->nullable(); // stored as raw SQL type below
    $table->timestamps();
});

// Raw statement for the vector column — Blueprint has no native type yet
DB::statement('ALTER TABLE document_chunks ADD COLUMN IF NOT EXISTS embedding vector(1536)');
DB::statement('CREATE INDEX ON document_chunks USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100)');

The ivfflat index trades perfect recall for speed. For datasets under ~100 k rows, an exact scan is fine — drop the index and benchmark before adding it.

2. Generating and Storing Embeddings

Wrap the OpenAI embeddings endpoint in a focused service:

final class EmbeddingService
{
    public function __construct(private readonly OpenAIClient $client) {}

    /** @return float[] */
    public function embed(string $text): array
    {
        $response = $this->client->embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $text,
        ]);

        return $response->embeddings[0]->embedding;
    }
}

Store chunks via a queued job so you never block an HTTP request:

final class StoreDocumentChunks implements ShouldQueue
{
    use Dispatchable, Queueable;

    public function __construct(private readonly int $documentId, private readonly string $text) {}

    public function handle(EmbeddingService $embedder, ChunkSplitter $splitter): void
    {
        foreach ($splitter->split($this->text, maxTokens: 400) as $chunk) {
            $vector = $embedder->embed($chunk);
            $literal = '[' . implode(',', $vector) . ']';

            DB::table('document_chunks')->insert([
                'document_id' => $this->documentId,
                'content'     => $chunk,
                'token_count' => count(explode(' ', $chunk)),
                'embedding'   => DB::raw("'$literal'::vector"),
                'created_at'  => now(),
                'updated_at'  => now(),
            ]);
        }
    }
}

3. Retrieval: Cosine Similarity Query

final class VectorRetriever
{
    public function __construct(private readonly EmbeddingService $embedder) {}

    /** @return Collection<int, object{content: string, similarity: float}> */
    public function retrieve(string $query, int $topK = 5): Collection
    {
        $vector = $this->embedder->embed($query);
        $literal = '[' . implode(',', $vector) . ']';

        return DB::table('document_chunks')
            ->selectRaw('content, 1 - (embedding <=> ?::vector) AS similarity', [$literal])
            ->orderByDesc('similarity')
            ->limit($topK)
            ->get();
    }
}

The <=> operator is pgvector's cosine distance. Subtracting from 1 gives similarity. No ORM magic needed — raw query builder keeps it transparent.

4. Assembling the Prompt

final class RagPipeline
{
    public function __construct(
        private readonly VectorRetriever $retriever,
        private readonly OpenAIClient $client,
    ) {}

    public function answer(string $question): string
    {
        $chunks = $this->retriever->retrieve($question, topK: 5);

        $context = $chunks
            ->map(fn ($row) => $row->content)
            ->implode("\n\n---\n\n");

        $response = $this->client->chat()->create([
            'model'    => 'gpt-4o-mini',
            'messages' => [
                ['role' => 'system', 'content' => "Answer using only the context below.\n\n$context"],
                ['role' => 'user',   'content' => $question],
            ],
        ]);

        return $response->choices[0]->message->content;
    }
}

Bind RagPipeline in a service provider and inject it into a controller or Artisan command. The whole pipeline is testable: fake EmbeddingService to return a fixed vector, seed document_chunks, assert the SQL similarity query runs.

Key Takeaways

• pgvector + PostgreSQL eliminates the need for a dedicated vector database at moderate scale.
• Queue embedding jobs — never block HTTP workers on synchronous embedding API calls.
• ivfflat index is a good default; add it only after you have enough rows to justify the build cost.
• Cosine distance (<=>) works well for text embeddings; use <#> (inner product) only when vectors are normalized.
• Keep retrieval and generation separate — VectorRetriever and RagPipeline are independently testable and swappable.
• Chunk size matters: 300–500 tokens per chunk balances recall and context window usage for most models.