OpenAI embeddings, pgvector in PostgreSQL, cosine similarity queries in Eloquent, chunking your content for indexing, hybrid keyword + semantic search, and the caching strategy that keeps your embedding costs near zero — a complete implementation that your users will notice on day one.
A user searches “why won’t my payment go through” in a support docs site. The article that answers this exact question is titled “Resolving Declined Card Errors.” Zero shared keywords. LIKE '%payment%won%through%' returns nothing, because there’s nothing to match — the words in the query and the words in the answer are different words describing the same problem. This is the ceiling every keyword search system hits, and it’s not a tuning problem. Full-text search, however well-indexed, matches text. It doesn’t know that “won’t go through” and “declined” mean the same thing to the person asking.
Semantic search solves exactly this — and as of 2026 it doesn’t require a dedicated vector database, a separate service, or a large infrastructure investment. PostgreSQL’s pgvector extension turns your existing database into a vector store. OpenAI’s text-embedding-3-small model costs $0.02 per million tokens for indexing — embedding an entire mid-sized documentation site costs cents, not dollars. This post is the complete implementation: chunking content correctly, generating and storing embeddings, querying by cosine similarity through Eloquent, combining semantic and keyword search so neither one’s weaknesses dominate, and the caching layer that keeps query-time embedding costs near zero.
The Architecture, Before Any Code
Three pieces, and understanding what each one does before writing code prevents most of the mistakes people make implementing this the first time:
- Chunking — breaking source content into pieces small enough to embed meaningfully and retrieve precisely. A whole 3,000-word article embedded as one vector produces a mediocre, averaged-out representation of everything it discusses. Chunked into sections, each chunk’s embedding represents one specific idea, and retrieval returns the actual relevant passage, not just the right document.
- Embedding + storage — converting each chunk into a vector (a list of 1,536 numbers, for
text-embedding-3-small) that captures its meaning, stored in Postgres viapgvectoralongside the chunk’s text and a reference back to its source. - Query-time retrieval — embedding the user’s search query the same way, then finding the stored chunks whose vectors are closest to it by cosine similarity, optionally blended with a traditional keyword search.
None of these three steps is optional, and skipping straight to “call the embeddings API and store the result” without a real chunking strategy is the most common reason a first implementation returns technically-relevant-but-practically-useless results.
Setting Up pgvector
-- Run once, requires the pgvector extension installed on the Postgres server
CREATE EXTENSION IF NOT EXISTS vector;
Most managed Postgres providers (RDS, Supabase, Neon, DigitalOcean) support pgvector as an enable-able extension without any server-level installation. If self-hosting, it needs to be compiled and installed on the Postgres instance itself before the migration below will run.
// database/migrations/xxxx_create_document_chunks_table.php
public function up(): void
{
DB::statement('CREATE EXTENSION IF NOT EXISTS vector');
Schema::create('document_chunks', function (Blueprint $table) {
$table->id();
$table->foreignId('document_id')->constrained()->cascadeOnDelete();
$table->text('content');
$table->integer('chunk_index');
$table->timestamps();
});
// pgvector's column type isn't a native Laravel Blueprint method — raw SQL
DB::statement('ALTER TABLE document_chunks ADD COLUMN embedding vector(1536)');
// HNSW index — the right default for query-time speed at this scale
DB::statement('CREATE INDEX document_chunks_embedding_idx ON document_chunks
USING hnsw (embedding vector_cosine_ops)');
}
vector(1536) matches text-embedding-3-small‘s output dimensions exactly — this has to match the embedding model’s actual output size, or every insert fails. The hnsw index type is the right default for most applications: it trades a small amount of recall accuracy for significantly faster approximate nearest-neighbor queries compared to ivfflat, and unlike ivfflat, it doesn’t need to be rebuilt as the table grows — ivfflat‘s index quality degrades as more rows are added unless it’s periodically reindexed, which is one more piece of operational overhead this setup doesn’t need.
Chunking Content Correctly
// app/Services/ContentChunker.php
class ContentChunker
{
public function __construct(
private int $maxTokens = 500,
private int $overlapTokens = 50,
) {}
public function chunk(string $content): array
{
$paragraphs = preg_split('/\n\s*\n/', trim($content));
$chunks = [];
$current = '';
foreach ($paragraphs as $paragraph) {
$combined = $current === '' ? $paragraph : "{$current}\n\n{$paragraph}";
if ($this->estimateTokens($combined) > $this->maxTokens && $current !== '') {
$chunks[] = trim($current);
$current = $this->tail($current) . "\n\n" . $paragraph;
} else {
$current = $combined;
}
}
if (trim($current) !== '') {
$chunks[] = trim($current);
}
return $chunks;
}
private function tail(string $text): string
{
// Carry the last ~overlapTokens worth of words forward into the next
// chunk, so a sentence that got cut at a chunk boundary isn't
// stripped of the context immediately before it.
$words = explode(' ', $text);
return implode(' ', array_slice($words, -$this->overlapTokens));
}
private function estimateTokens(string $text): int
{
// ~4 characters per token is a reasonable estimate for English text
// without pulling in a full tokenizer for a chunking pass.
return (int) ceil(strlen($text) / 4);
}
}
Chunk by paragraph boundary, not by fixed character count. Splitting at an arbitrary character offset regularly cuts a sentence in half between two chunks, and the embedding for a fragment like “the payment fails because the card issuer” (with the actual reason cut off into the next chunk) captures a meaningfully different — and less useful — meaning than the complete sentence would have.
The overlap matters more than it looks like it should. Without it, a fact stated right at a chunk boundary can end up split so that neither chunk fully contains it, and a query matching that fact retrieves neither chunk strongly. Fifty tokens of carried-forward context is cheap — a small percentage increase in total tokens embedded — and it closes this gap.
500 tokens is a reasonable default, not a universal constant. Short FAQ-style content might chunk better at 200–300 tokens per chunk, so each chunk maps to one question-answer pair precisely. Long-form technical documentation with multi-paragraph explanations might do better at 750–1000, so a chunk isn’t cut in the middle of a single coherent explanation. The number worth testing against real queries from real users, not guessing once and leaving alone.
Generating and Storing Embeddings
// app/Services/EmbeddingService.php
class EmbeddingService
{
public function __construct(private OpenAI $client) {}
public function embed(string $text): array
{
$response = $this->client->embeddings()->create([
'model' => 'text-embedding-3-small',
'input' => $text,
]);
return $response->embeddings[0]->embedding;
}
public function embedBatch(array $texts): array
{
// One API call for many chunks — the OpenAI embeddings endpoint
// accepts an array input, so batching indexing work into a single
// request is both faster and avoids per-call overhead.
$response = $this->client->embeddings()->create([
'model' => 'text-embedding-3-small',
'input' => $texts,
]);
return array_map(fn ($item) => $item->embedding, $response->embeddings);
}
}
// app/Jobs/IndexDocumentJob.php
class IndexDocumentJob implements ShouldQueue
{
use Queueable;
public function __construct(private Document $document) {}
public function handle(ContentChunker $chunker, EmbeddingService $embeddings): void
{
// Re-indexing: clear old chunks first so a content edit doesn't
// leave stale chunks alongside the new ones, doubling up matches.
$this->document->chunks()->delete();
$chunks = $chunker->chunk($this->document->body);
$vectors = $embeddings->embedBatch($chunks);
foreach ($chunks as $index => $content) {
DocumentChunk::create([
'document_id' => $this->document->id,
'content' => $content,
'chunk_index' => $index,
'embedding' => '[' . implode(',', $vectors[$index]) . ']',
]);
}
}
}
// app/Observers/DocumentObserver.php
class DocumentObserver
{
public function saved(Document $document): void
{
if ($document->wasChanged('body')) {
IndexDocumentJob::dispatch($document);
}
}
}
Indexing runs as a queued job, not inline on save — embedding a long document is several API round trips plus batch processing time, and blocking a request-response cycle on that is the kind of thing that turns an admin’s “save” click into a multi-second hang for no reason the admin can see. The observer dispatching only wasChanged('body') avoids re-embedding (and re-paying for) documents on every save when only unrelated metadata changed.
The '[' . implode(',', $vector) . ']' construction is deliberate — pgvector‘s column type expects a bracketed, comma-separated string on insert through raw values, since Eloquent doesn’t have native casting for the vector type.
Querying by Cosine Similarity Through Eloquent
// app/Models/DocumentChunk.php
class DocumentChunk extends Model
{
protected $fillable = ['document_id', 'content', 'chunk_index', 'embedding'];
public function document(): BelongsTo
{
return $this->belongsTo(Document::class);
}
public function scopeSimilarTo(Builder $query, array $vector, int $limit = 10): Builder
{
$vectorString = '[' . implode(',', $vector) . ']';
return $query
->select('*')
->selectRaw('1 - (embedding <=> ?) as similarity', [$vectorString])
->orderByRaw('embedding <=> ?', [$vectorString])
->limit($limit);
}
}
// app/Services/SemanticSearchService.php
class SemanticSearchService
{
public function __construct(private EmbeddingService $embeddings) {}
public function search(string $query, int $limit = 10): Collection
{
$vector = $this->embeddings->embed($query);
return DocumentChunk::query()
->similarTo($vector, $limit)
->with('document')
->get();
}
}
<=> is pgvector‘s cosine distance operator — 0 means identical direction (maximally similar), 2 means opposite. 1 - (embedding <=> ?) converts that distance into a similarity score in the more intuitive 0-to-1-descending-to-negative range, so ordering by it descending puts the most relevant chunks first without the caller needing to know the underlying distance math. Ordering by the raw <=> expression directly (ascending) in the query itself, rather than computing similarity in PHP after fetching, is what lets the HNSW index actually do its job — Postgres can use the index to find nearest neighbors efficiently precisely because the ordering is expressed as a distance operation it understands natively.
$results = app(SemanticSearchService::class)->search('why is my card getting declined');
foreach ($results as $chunk) {
echo "{$chunk->document->title} (similarity: " . round($chunk->similarity, 3) . ")\n";
echo "{$chunk->content}\n\n";
}
Hybrid Search — Where Semantic Search Alone Falls Short
Semantic search is worse than keyword search at exactly the cases keyword search is best at: exact product codes, specific error strings, proper nouns, SKUs — anything where the literal characters matter more than the meaning. A user searching for the exact error code ERR_PAYMENT_declined_4021 wants the document containing that exact string, and an embedding model has no special reason to rank it above a semantically-similar-but-textually-different chunk.
// app/Services/HybridSearchService.php
class HybridSearchService
{
public function __construct(
private EmbeddingService $embeddings,
) {}
public function search(string $query, int $limit = 10): Collection
{
$vector = $this->embeddings->embed($query);
$vectorString = '[' . implode(',', $vector) . ']';
// Reciprocal Rank Fusion — combine two independently ranked result
// sets without needing to normalize two different scoring scales
// against each other.
return DocumentChunk::query()
->selectRaw('
document_chunks.*,
(
COALESCE(1.0 / (60 + semantic_rank.rank), 0) +
COALESCE(1.0 / (60 + keyword_rank.rank), 0)
) as combined_score
', [])
->joinSub(
DocumentChunk::query()
->selectRaw('id, ROW_NUMBER() OVER (ORDER BY embedding <=> ?) as rank', [$vectorString])
->orderByRaw('embedding <=> ?', [$vectorString])
->limit(50),
'semantic_rank',
'document_chunks.id',
'=',
'semantic_rank.id'
)
->leftJoinSub(
DocumentChunk::query()
->selectRaw("id, ROW_NUMBER() OVER (ORDER BY ts_rank(to_tsvector('english', content), plainto_tsquery('english', ?)) DESC) as rank", [$query])
->whereRaw("to_tsvector('english', content) @@ plainto_tsquery('english', ?)", [$query])
->limit(50),
'keyword_rank',
'document_chunks.id',
'=',
'keyword_rank.id'
)
->orderByDesc('combined_score')
->with('document')
->limit($limit)
->get();
}
}
Reciprocal Rank Fusion (RRF) combines two ranked lists — the semantic similarity ranking and Postgres’s own full-text search ranking — into one score, without needing to normalize cosine similarity and ts_rank‘s scoring onto a shared scale, which is a much harder problem than it sounds like (the two scores don’t have comparable distributions, and naive averaging tends to let whichever score happens to have a wider numeric range dominate). RRF sidesteps that entirely by scoring based on rank position in each list, not the raw score — a chunk ranked #1 in both lists scores highly regardless of what the underlying similarity or ts_rank numbers actually were. The 60 constant is a standard RRF smoothing value that prevents a #1 rank in one list from completely dominating a #2 rank in the other; it doesn’t need tuning for most applications.
This is the setup worth reaching for once semantic-only search has shipped and the team notices the specific failure mode — exact-match queries (error codes, SKUs, names) returning worse results than a plain keyword search would have. Shipping hybrid search from the very first version is defensible too, but semantic-only first gets something meaningfully better than keyword search in front of users faster, with hybrid as the deliberate follow-up once the exact-match gap is a real, observed problem rather than a hypothetical one.
The Caching Strategy That Keeps Costs Near Zero
Indexing cost is a one-time (or content-change-triggered) expense — cents for most content sizes at text-embedding-3-small‘s $0.02-per-million-token rate. The cost that actually recurs is query-time embedding: every search request needs the query text embedded before it can be compared against stored vectors, and without caching, that’s one paid API call per search, forever, including the tenth time in a day that five different users search for some version of the same common question.
// app/Services/EmbeddingService.php — with caching added
class EmbeddingService
{
public function __construct(private OpenAI $client) {}
public function embed(string $text): array
{
$cacheKey = 'embedding:' . hash('sha256', mb_strtolower(trim($text)));
return Cache::remember($cacheKey, now()->addDays(30), function () use ($text) {
$response = $this->client->embeddings()->create([
'model' => 'text-embedding-3-small',
'input' => $text,
]);
return $response->embeddings[0]->embedding;
});
}
}
mb_strtolower(trim($text)) before hashing is what makes this cache actually hit in practice — “Why is my card declined” and “why is my card declined?” are different strings but represent the same search intent, and normalizing case and whitespace before hashing means both map to the same cache key instead of paying for two nearly-identical embeddings. A 30-day TTL is a reasonable default for query caching specifically — search queries have a long tail, but common questions repeat constantly, and there’s no reason to re-embed “how do I reset my password” every single time it’s typed.
For applications with genuinely high query volume, layering a normalized-query cache on top — mapping common query variations to a single canonical form before even checking the embedding cache — captures more hits than exact-string caching alone, though the added complexity is only worth it once query volume and repetition patterns justify it; for most applications, the straightforward hash-based cache above already eliminates the overwhelming majority of redundant embedding calls.
What caching does not need to cover: indexing embeddings. Those are generated once per chunk, stored permanently in the embedding column, and never re-requested unless the source content changes and triggers re-indexing — there’s no repeated cost there to cache against in the first place. The entire cost surface worth optimizing is query-time embedding calls, and a simple normalized cache closes nearly all of it.
The Complete Flow
Content saved/updated
→ Observer detects change
→ IndexDocumentJob queued
→ ContentChunker splits into paragraph-bounded, overlapping chunks
→ EmbeddingService.embedBatch() — one API call for all chunks
→ Chunks + vectors stored in document_chunks via pgvector
User searches
→ Query normalized (lowercase, trimmed) and hashed
→ Cache checked — hit avoids an API call entirely
→ On miss: EmbeddingService.embed() — one API call, cached for 30 days
→ HybridSearchService: semantic rank (HNSW-indexed cosine distance)
+ keyword rank (Postgres full-text search) combined via RRF
→ Results returned, ordered by combined_score
The One Rule
Semantic search fails in production for one of three reasons, almost always: content chunked at arbitrary boundaries instead of paragraph boundaries, so embeddings represent fragments instead of ideas; no keyword fallback, so exact-match queries like error codes and SKUs underperform a plain LIKE query; or no query-time caching, so a proof-of-concept that worked fine at ten searches a day turns into an unexpectedly large embeddings bill at ten thousand. None of the three is expensive to fix. All three are expensive to discover after users are already relying on search that’s quietly worse — or quietly more expensive — than it needed to be.