Practical RAG in Laravel: pgvector, Embeddings, and Retrieval Pipelines

Why RAG Instead of Fine-Tuning?

Fine-tuning a model on your domain data is expensive, slow to iterate, and quickly goes stale. Retrieval-Augmented Generation (RAG) keeps your knowledge base in a vector store and fetches only the relevant chunks at query time — giving the LLM fresh, scoped context without retraining.

This article walks through a practical Laravel implementation: storing embeddings in PostgreSQL via pgvector, querying them with a raw Eloquent expression, and wiring everything into a clean service that your controllers and jobs can call.

1. Enable pgvector in PostgreSQL

Install the extension once per database:

CREATE EXTENSION IF NOT EXISTS vector;

Then create a migration for your chunks table:

Schema::create('document_chunks', function (Blueprint $table) {
    $table->id();
    $table->foreignId('document_id')->constrained()->cascadeOnDelete();
    $table->text('content');
    $table->string('model')->default('text-embedding-3-small');
    // 1536 dims for text-embedding-3-small
    $table->vector('embedding', 1536)->nullable();
    $table->timestamps();
});

Laravel's Blueprint doesn't know vector natively, so register a macro in a service provider:

use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

Blueprint::macro('vector', function (string $column, int $dimensions) {
    return $this->addColumn('vector', $column, compact('dimensions'));
});

And register the custom type with Doctrine in AppServiceProvider:

DB::connection()->getDoctrineSchemaManager()
    ->getDatabasePlatform()
    ->registerDoctrineTypeMapping('vector', 'string');

2. Generating and Storing Embeddings

Create an EmbeddingService that wraps the OpenAI HTTP call:

final readonly class EmbeddingService
{
    public function __construct(
        private \OpenAI\Client $client,
        private string $model = 'text-embedding-3-small',
    ) {}

    /** @return float[] */
    public function embed(string $text): array
    {
        $response = $this->client->embeddings()->create([
            'model' => $this->model,
            'input' => $text,
        ]);

        return $response->embeddings[0]->embedding;
    }
}

When a document is ingested, chunk it and dispatch a job:

final class EmbedChunkJob implements ShouldQueue
{
    use Dispatchable, Queueable;

    public function __construct(private readonly int $chunkId) {}

    public function handle(EmbeddingService $embeddings): void
    {
        $chunk = DocumentChunk::findOrFail($this->chunkId);
        $vector = $embeddings->embed($chunk->content);

        // pgvector expects a bracketed string: '[0.1,0.2,...]'
        $chunk->update(['embedding' => '[' . implode(',', $vector) . ']']);
    }
}

3. Similarity Search with Eloquent

Cosine distance (<=>) is the right operator for normalized OpenAI embeddings:

final class ChunkRepository
{
    public function nearest(array $queryVector, int $limit = 5): Collection
    {
        $literal = '[' . implode(',', $queryVector) . ']';

        return DocumentChunk::query()
            ->selectRaw('*, embedding <=> ? AS distance', [$literal])
            ->whereNotNull('embedding')
            ->orderByRaw('embedding <=> ?', [$literal])
            ->limit($limit)
            ->get();
    }
}

Add an IVFFlat index for datasets beyond ~100k rows:

CREATE INDEX ON document_chunks
    USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 100);

4. Wiring the RAG Pipeline

final readonly class RagService
{
    public function __construct(
        private EmbeddingService $embeddings,
        private ChunkRepository $chunks,
        private \OpenAI\Client $client,
    ) {}

    public function answer(string $question): string
    {
        $vector = $this->embeddings->embed($question);
        $context = $this->chunks->nearest($vector)
            ->pluck('content')
            ->implode("\n\n---\n\n");

        $response = $this->client->chat()->create([
            'model' => 'gpt-4o-mini',
            'messages' => [
                ['role' => 'system', 'content' => "Answer using only the context below.\n\n{$context}"],
                ['role' => 'user',   'content' => $question],
            ],
        ]);

        return $response->choices[0]->message->content;
    }
}

Bind it in a service provider and inject it wherever needed — controllers, Livewire components, or Filament actions.

Key Takeaways

• pgvector keeps your vector store inside Postgres — no extra infrastructure.
• Use a Blueprint macro to add the vector column type cleanly in migrations.
• Cosine distance (<=>) works best with OpenAI's normalized embeddings.
• Offload embedding generation to queued jobs to avoid blocking HTTP requests.
• An IVFFlat index is essential once your chunk count grows beyond tens of thousands.
• Keep the RAG logic in a dedicated RagService — controllers stay thin and the pipeline stays testable.