Why Semantic Search Beats LIKE Queries
Full-text search handles keywords. Semantic search handles meaning. When a user types affordable accommodation near the beach, they do not want rows containing those exact words — they want conceptually relevant results. PostgreSQL's pgvector extension, combined with OpenAI embeddings, makes this achievable without leaving your existing Laravel and Postgres stack.
1. Enable pgvector
On a self-hosted Postgres instance run: CREATE EXTENSION IF NOT EXISTS vector;
For Laravel Forge or Ploi, SSH in and run sudo apt install postgresql-15-pgvector then the SQL above. Most managed providers such as Supabase, Neon, and Railway already ship it.
2. Migration: Add a Vector Column
Laravel's Blueprint does not know the vector type natively, so use addColumn with the raw type name. pgvector registers it with Postgres directly. Add an HNSW index for fast approximate nearest-neighbour queries using vector_cosine_ops.
3. Generating and Storing Embeddings
Create a dedicated EmbeddingService rather than scattering API calls across your codebase. Call OpenAI::embeddings()->create() with model text-embedding-3-small and store the resulting float array. pgvector expects the literal string format [0.12,0.34,...] so implode the array with commas and wrap it in brackets before saving.
Store the embedding inside a queued job triggered by a model observer so article saves stay fast and do not block the request cycle.
4. Querying by Similarity
Create an ArticleSearchRepository that accepts a plain-text query, converts it to an embedding via EmbeddingService, then queries the database using selectRaw with the cosine distance operator. The pgvector operator for cosine distance is the spaceship-arrow operator. Subtracting the distance from 1 gives cosine similarity where higher values mean more relevant results. Order by similarity descending and limit to the desired number of results.
5. Chunking Long Documents
OpenAI embedding models have token limits. For long articles, split the text into word chunks of around 500 words each. Store each chunk as a separate row in an article_chunks table with its own embedding column and a foreign key back to articles. Your search query then joins back to the parent article and deduplicates results.
Key Takeaways
- Use addColumn with the vector type in migrations because Laravel Blueprint has no native vector type yet.
- HNSW indexes with vector_cosine_ops give sub-millisecond approximate search at scale. Build them after bulk-inserting embeddings.
- Normalise and truncate text before sending to the API to avoid token overflows and reduce cost.
- Wrap embedding generation in a queued job so article saves stay fast.
- Cosine distance works best for text embeddings. Use L2 distance for image or numeric vectors.
