Practical RAG in Laravel: pgvector, Embeddings, and Retrieval-Augmented Generation

Why RAG Instead of Fine-Tuning?

Fine-tuning is expensive and goes stale. Retrieval-augmented generation (RAG) keeps your knowledge base in a database you already control, lets you update it without retraining, and gives you full auditability over what context the model sees. If you are already running PostgreSQL, the pgvector extension turns it into a capable vector store — no extra infrastructure required.

Enabling pgvector in a Laravel Migration

// database/migrations/2024_01_01_000000_create_document_chunks_table.php
public function up(): void
{
    DB::statement('CREATE EXTENSION IF NOT EXISTS vector');

    Schema::create('document_chunks', function (Blueprint $table) {
        $table->id();
        $table->foreignId('document_id')->constrained()->cascadeOnDelete();
        $table->text('content');
        $table->integer('token_count');
        // 1536 dimensions for text-embedding-3-small
        $table->string('embedding', 8192)->nullable(); // stored as text, cast in queries
        $table->timestamps();
    });

    DB::statement(
        'ALTER TABLE document_chunks ADD COLUMN embedding_vec vector(1536)'
    );

    // IVFFlat index — tune lists to ~sqrt(row_count)
    DB::statement(
        'CREATE INDEX ON document_chunks USING ivfflat (embedding_vec vector_cosine_ops) WITH (lists = 100)'
    );
}

Using a raw ALTER TABLE keeps the migration readable while sidestepping Blueprint's lack of native vector type support.

Generating and Storing Embeddings

Wrap the OpenAI call in a dedicated action so it is easy to swap providers later.

namespace App\AI;

use App\Models\DocumentChunk;
use OpenAI\Laravel\Facades\OpenAI;

final class EmbedChunk
{
    public function handle(DocumentChunk $chunk): void
    {
        $response = OpenAI::embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $chunk->content,
        ]);

        $vector = $response->embeddings[0]->embedding; // float[]

        // pgvector expects '[0.1,0.2,...]' literal syntax
        $literal = '[' . implode(',', $vector) . ']';

        $chunk->updateQuietly(['embedding_vec' => DB::raw("'$literal'::vector")]);
    }
}

Dispatch this inside a queued job after chunking a document so the HTTP cycle stays fast.

Retrieval: Nearest-Neighbour Search

namespace App\AI;

use App\Models\DocumentChunk;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\DB;

final class RetrieveChunks
{
    public function __construct(private EmbedQuery $embedQuery) {}

    /** @return Collection<int, DocumentChunk> */
    public function topK(string $query, int $k = 5): Collection
    {
        $vector = $this->embedQuery->forText($query);
        $literal = '[' . implode(',', $vector) . ']';

        return DocumentChunk::query()
            ->select(['id', 'document_id', 'content', 'token_count'])
            ->selectRaw(
                'embedding_vec <=> ?::vector AS distance',
                [$literal]
            )
            ->orderBy('distance')
            ->limit($k)
            ->get();
    }
}

The <=> operator is cosine distance. Use <#> for inner product or <-> for L2 depending on how your embeddings were normalised.

Assembling the Prompt

final class RagPipeline
{
    public function __construct(
        private RetrieveChunks $retriever,
    ) {}

    public function answer(string $question): string
    {
        $chunks = $this->retriever->topK($question, k: 5);

        $context = $chunks
            ->map(fn ($c) => "---\n" . $c->content)
            ->implode("\n");

        $response = OpenAI::chat()->create([
            'model' => 'gpt-4o-mini',
            'messages' => [
                ['role' => 'system', 'content' =>
                    'Answer using only the context below. If unsure, say so.'],
                ['role' => 'user', 'content' =>
                    "Context:\n$context\n\nQuestion: $question"],
            ],
            'max_tokens' => 512,
        ]);

        return $response->choices[0]->message->content;
    }
}

Chunking Strategy Matters

A naive split on paragraph breaks produces uneven chunks. Prefer a sliding-window approach: 512-token chunks with a 64-token overlap so context is not lost at boundaries. Count tokens with tiktoken-php before persisting.

$chunks = collect($splitter->split($text, chunkSize: 512, overlap: 64));

Production Considerations

• Index warm-up: IVFFlat requires SET ivfflat.probes = 10 at query time for better recall at the cost of speed. Tune per workload.
• Embedding cache: Hash the input text and cache the vector in Redis to avoid redundant API calls during re-indexing.
• Tenant isolation: Add a tenant_id column and include it in the WHERE clause before the vector search; pgvector will filter first if the planner cooperates.
• Batch embedding: Use the embeddings endpoint's array input to embed up to 2048 strings per request and cut API round-trips dramatically.

Takeaways

• pgvector + PostgreSQL is a viable production vector store for most Laravel apps — no separate service needed.
• Keep embedding generation in queued jobs; keep retrieval in a dedicated class injectable via the service container.
• The <=> cosine operator, IVFFlat index, and probes setting are the three knobs that control recall vs. latency.
• Chunking quality directly determines answer quality — invest time in your splitter before tuning the model.
• Cache embeddings aggressively; the embedding API is the main cost driver in a RAG pipeline.