Practical RAG in Laravel: pgvector, Embeddings, and Retrieval Pipelines

Why RAG in Laravel Without a Dedicated Vector DB?

Pinecone, Weaviate, and Qdrant are compelling, but they add operational overhead. If you are already running PostgreSQL, the pgvector extension gives you cosine-similarity search with a single CREATE EXTENSION and a new column type. For most SaaS workloads — knowledge bases, document Q&A, semantic search — that is more than enough.

This article walks through a complete, opinionated RAG pipeline: ingestion, embedding storage, retrieval, and prompt assembly.

1. Enable pgvector and Add the Column

CREATE EXTENSION IF NOT EXISTS vector;

// database/migrations/2024_06_01_create_document_chunks_table.php
Schema::create('document_chunks', function (Blueprint $table) {
    $table->id();
    $table->foreignId('document_id')->constrained()->cascadeOnDelete();
    $table->text('content');
    $table->vector('embedding', 1536); // OpenAI text-embedding-3-small
    $table->timestamps();
});

Laravel's Blueprint does not ship a vector() macro out of the box, so register one in a service provider:

use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\DB;

Blueprint::macro('vector', function (string $column, int $dimensions = 1536) {
    /** @var Blueprint $this */
    return $this->addColumn('vector', $column, compact('dimensions'));
});

// In AppServiceProvider::boot()
\Illuminate\Database\Schema\Grammars\PostgresGrammar::macro(
    'typeVector',
    fn ($column) => "vector({$column->dimensions})"
);

2. The Embedding Service

Keep the OpenAI HTTP call behind a thin interface so you can swap providers or stub in tests.

interface EmbeddingProvider
{
    /** @return float[] */
    public function embed(string $text): array;
}

final class OpenAiEmbeddingProvider implements EmbeddingProvider
{
    public function __construct(
        private readonly \OpenAI\Client $client,
        private readonly string $model = 'text-embedding-3-small',
    ) {}

    public function embed(string $text): array
    {
        $response = $this->client->embeddings()->create([
            'model' => $this->model,
            'input' => $text,
        ]);

        return $response->embeddings[0]->embedding;
    }
}

Bind it in the container:

$this->app->singleton(EmbeddingProvider::class, fn () =>
    new OpenAiEmbeddingProvider(app(\OpenAI\Client::class))
);

3. Ingestion Pipeline

Chunk documents before embedding — 512-token chunks with a 64-token overlap is a solid default.

final class IngestDocumentAction
{
    public function __construct(
        private readonly EmbeddingProvider $embedder,
    ) {}

    public function handle(Document $document): void
    {
        $chunks = $this->chunk($document->body, maxTokens: 512, overlap: 64);

        foreach ($chunks as $content) {
            $vector = $this->embedder->embed($content);

            DB::table('document_chunks')->insert([
                'document_id' => $document->id,
                'content'     => $content,
                'embedding'   => '[' . implode(',', $vector) . ']',
                'created_at'  => now(),
                'updated_at'  => now(),
            ]);
        }
    }

    /** @return string[] */
    private function chunk(string $text, int $maxTokens, int $overlap): array
    {
        // Naive word-boundary chunking; replace with tiktoken FFI for precision.
        $words  = explode(' ', $text);
        $chunks = [];
        $step   = max(1, $maxTokens - $overlap);

        for ($i = 0; $i < count($words); $i += $step) {
            $chunks[] = implode(' ', array_slice($words, $i, $maxTokens));
        }

        return array_filter($chunks);
    }
}

4. Retrieval: Cosine Similarity with pgvector

final class RetrievalService
{
    public function __construct(
        private readonly EmbeddingProvider $embedder,
    ) {}

    /** @return array<int, array{content: string, score: float}> */
    public function retrieve(string $query, int $topK = 5): array
    {
        $vector = '[' . implode(',', $this->embedder->embed($query)) . ']';

        return DB::select(
            "SELECT content,
                    1 - (embedding <=> ?::vector) AS score
             FROM document_chunks
             ORDER BY embedding <=> ?::vector
             LIMIT ?",
            [$vector, $vector, $topK]
        );
    }
}

The <=> operator is pgvector's cosine distance. Subtract from 1 to get similarity. Add an ivfflat index once your chunk count exceeds ~100k rows:

CREATE INDEX ON document_chunks
    USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 100);

5. Prompt Assembly and Generation

final class AnswerQuestionAction
{
    public function __construct(
        private readonly RetrievalService $retrieval,
        private readonly \OpenAI\Client $client,
    ) {}

    public function handle(string $question): string
    {
        $chunks  = $this->retrieval->retrieve($question, topK: 4);
        $context = collect($chunks)
            ->pluck('content')
            ->implode("\n\n---\n\n");

        $response = $this->client->chat()->create([
            'model'    => 'gpt-4o-mini',
            'messages' => [
                ['role' => 'system', 'content' =>
                    "Answer using only the context below.\n\n{$context}"],
                ['role' => 'user', 'content' => $question],
            ],
        ]);

        return $response->choices[0]->message->content;
    }
}

Key Takeaways

• pgvector is production-ready for most SaaS RAG workloads; skip the dedicated vector DB until you hit scale.
• Chunk with overlap to avoid splitting context across boundaries; tune chunk size per domain.
• Abstract the embedding provider behind an interface — it makes testing trivial and provider swaps painless.
• Use ivfflat indexes once chunk counts grow; lists = sqrt(row_count) is a reasonable starting heuristic.
• Keep retrieval and generation separate actions — retrieval is deterministic and cacheable; generation is not.