Practical RAG in Laravel: pgvector, Embeddings, and Retrieval Pipelines

Why RAG Instead of Fine-Tuning?

Retrieval-augmented generation (RAG) lets you ground an LLM's answers in your own data without the cost and complexity of fine-tuning. The pattern is simple: embed your documents, store the vectors, retrieve the top-k nearest neighbours at query time, and inject them as context. Laravel's ecosystem — PostgreSQL, Eloquent, and the HTTP client — is more than enough to build this cleanly.

Setting Up pgvector

Install the extension and add a migration:

CREATE EXTENSION IF NOT EXISTS vector;

// database/migrations/2024_01_01_000000_add_embedding_to_documents.php
public function up(): void
{
    Schema::table('documents', function (Blueprint $table) {
        // 1536 dims for text-embedding-3-small
        $table->vector('embedding', 1536)->nullable();
    });

    DB::statement(
        'CREATE INDEX documents_embedding_hnsw_idx
         ON documents USING hnsw (embedding vector_cosine_ops)'
    );
}

Laravel's Blueprint doesn't know vector natively, so register a macro in a service provider:

use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

Blueprint::macro('vector', function (string $column, int $dimensions): \Illuminate\Database\Schema\ColumnDefinition {
    return $this->addColumn('vector', $column, compact('dimensions'));
});

// Register the type with Doctrine so migrations don't break
\Doctrine\DBAL\Types\Type::addType('vector', VectorType::class);

VectorType is a thin Doctrine type that serialises a PHP float array to the [0.1,0.2,...] string pgvector expects.

Generating and Storing Embeddings

Wrap the OpenAI call in a dedicated action:

final readonly class GenerateEmbedding
{
    public function __construct(private \Illuminate\Http\Client\Factory $http) {}

    /** @return float[] */
    public function handle(string $text): array
    {
        $response = $this->http
            ->withToken(config('services.openai.key'))
            ->post('https://api.openai.com/v1/embeddings', [
                'model' => 'text-embedding-3-small',
                'input' => $text,
            ])
            ->throw()
            ->json('data.0.embedding');

        return $response;
    }
}

Dispatch a job when a document is saved:

final class EmbedDocument implements ShouldQueue
{
    use Dispatchable, Queueable;

    public function __construct(public readonly int $documentId) {}

    public function handle(GenerateEmbedding $action): void
    {
        $doc = Document::findOrFail($this->documentId);
        $vector = $action->handle($doc->body);

        // Store as pgvector literal
        DB::table('documents')
            ->where('id', $doc->id)
            ->update(['embedding' => '[' . implode(',', $vector) . ']']);
    }
}

Retrieval: Nearest-Neighbour Query

A clean retrieval abstraction keeps the pgvector SQL out of your controllers:

final readonly class DocumentRetriever
{
    public function __construct(
        private GenerateEmbedding $embedder,
        private int $topK = 5,
    ) {}

    /** @return \Illuminate\Support\Collection<int, Document> */
    public function retrieve(string $query): \Illuminate\Support\Collection
    {
        $vector = '[' . implode(',', $this->embedder->handle($query)) . ']';

        return Document::query()
            ->selectRaw('*, embedding <=> ? AS distance', [$vector])
            ->whereNotNull('embedding')
            ->orderBy('distance')
            ->limit($this->topK)
            ->get();
    }
}

The <=> operator is pgvector's cosine distance. For inner-product similarity use <#>.

Building the Prompt

final readonly class RagPipeline
{
    public function __construct(
        private DocumentRetriever $retriever,
        private \Illuminate\Http\Client\Factory $http,
    ) {}

    public function answer(string $question): string
    {
        $context = $this->retriever->retrieve($question)
            ->map(fn (Document $d) => "- {$d->title}: {$d->body}")
            ->implode("\n");

        return $this->http
            ->withToken(config('services.openai.key'))
            ->post('https://api.openai.com/v1/chat/completions', [
                'model' => 'gpt-4o-mini',
                'messages' => [
                    ['role' => 'system', 'content' => "Answer using only the context below.\n\n{$context}"],
                    ['role' => 'user',   'content' => $question],
                ],
            ])
            ->throw()
            ->json('choices.0.message.content');
    }
}

Caching Embeddings

Embedding the same query repeatedly wastes tokens. Cache by hash:

$cacheKey = 'embedding:' . hash('xxh128', $text);
$vector = Cache::remember($cacheKey, now()->addDay(), fn () => $action->handle($text));

Key Takeaways

• Register a Blueprint::macro for vector columns; pair it with a Doctrine type for migration compatibility.
• Use HNSW indexes (vector_cosine_ops) for sub-millisecond ANN at scale — IVFFlat requires a VACUUM before it becomes useful.
• Keep embedding generation in a queued job; retrieval at request time is fast enough for synchronous use.
• Wrap retrieval behind a DocumentRetriever class so you can swap pgvector for another store without touching controllers.
• Cache query embeddings by content hash to avoid redundant API calls on repeated questions.