Practical RAG in Laravel: pgvector, Embeddings, and Retrieval Pipelines

Why RAG Instead of Fine-Tuning?

Retrieval-augmented generation (RAG) lets you ground an LLM's answers in your own data without the cost and complexity of fine-tuning. The pattern is straightforward: embed your documents, store the vectors, embed the user query at runtime, retrieve the closest chunks, and inject them into the prompt. PostgreSQL's pgvector extension makes this viable without a dedicated vector store.

Setting Up pgvector in Laravel

Enable the extension in a migration:

public function up(): void
{
    DB::statement('CREATE EXTENSION IF NOT EXISTS vector');

    Schema::create('document_chunks', function (Blueprint $table) {
        $table->id();
        $table->foreignId('document_id')->constrained()->cascadeOnDelete();
        $table->text('content');
        $table->string('embedding_model', 64)->default('text-embedding-3-small');
        // Store as text; cast to vector in queries
        $table->text('embedding');
        $table->timestamps();
    });

    // Create an IVFFlat index after bulk-loading data
    DB::statement(
        'CREATE INDEX document_chunks_embedding_idx
         ON document_chunks
         USING ivfflat (embedding::vector(1536) vector_cosine_ops)
         WITH (lists = 100)'
    );
}

Note: IVFFlat requires data to exist before the index is useful. For smaller datasets (< 100k rows) an exact scan without an index is often fast enough.

Embedding Service

Wrap the OpenAI call behind an interface so you can swap providers or mock in tests:

interface EmbeddingProvider
{
    /** @return float[] */
    public function embed(string $text): array;
}

final class OpenAiEmbeddingProvider implements EmbeddingProvider
{
    public function __construct(
        private readonly OpenAI\Client $client,
        private readonly string $model = 'text-embedding-3-small',
    ) {}

    public function embed(string $text): array
    {
        $response = $this->client->embeddings()->create([
            'model' => $this->model,
            'input' => $text,
        ]);

        return $response->embeddings[0]->embedding;
    }
}

Bind it in a service provider:

$this->app->singleton(
    EmbeddingProvider::class,
    fn () => new OpenAiEmbeddingProvider(
        client: OpenAI::client(config('services.openai.key')),
    )
);

Ingestion Pipeline

Chunk documents and persist embeddings as a queued job:

final class IngestDocumentChunks implements ShouldQueue
{
    use Dispatchable, Queueable;

    public function __construct(private readonly Document $document) {}

    public function handle(EmbeddingProvider $embedder): void
    {
        $chunks = TextSplitter::splitByTokens($this->document->body, maxTokens: 512);

        foreach ($chunks as $content) {
            $vector = $embedder->embed($content);
            $literal = '[' . implode(',', $vector) . ']';

            DB::table('document_chunks')->insert([
                'document_id' => $this->document->id,
                'content'     => $content,
                'embedding'   => $literal,
                'created_at'  => now(),
                'updated_at'  => now(),
            ]);
        }
    }
}

Retrieval Service

At query time, embed the question and pull the top-k chunks by cosine similarity:

final class ChunkRetriever
{
    public function __construct(private readonly EmbeddingProvider $embedder) {}

    /** @return Collection<int, object{content: string, similarity: float}> */
    public function retrieve(string $query, int $topK = 5): Collection
    {
        $vector = $this->embedder->embed($query);
        $literal = '[' . implode(',', $vector) . ']';

        return DB::select(
            "SELECT content,
                    1 - (embedding::vector(1536) <=> ?::vector(1536)) AS similarity
             FROM document_chunks
             ORDER BY embedding::vector(1536) <=> ?::vector(1536)
             LIMIT ?",
            [$literal, $literal, $topK]
        ) |> collect(...);
    }
}

The <=> operator is cosine distance; subtracting from 1 gives similarity.

Prompt Assembly

$chunks = $retriever->retrieve($userQuestion);
$context = $chunks->pluck('content')->implode("\n\n---\n\n");

$messages = [
    ['role' => 'system', 'content' => "Answer using only the context below.\n\n{$context}"],
    ['role' => 'user',   'content' => $userQuestion],
];

Keep the system prompt tight. Stuffing too many chunks degrades answer quality and burns tokens.

Key Takeaways

• pgvector removes the need for a separate vector database for most Laravel applications.
• IVFFlat indexes trade recall for speed; tune lists and probes based on your dataset size.
• Interface-backed embedding providers make unit testing and provider swaps trivial.
• Chunk size matters: 256–512 tokens per chunk balances retrieval precision and context coverage.
• Cosine distance (<=>) is the right operator for normalized OpenAI embeddings.
• Queue ingestion jobs; embedding API calls are slow and should never block a request cycle.