From queued embeddings to typed AI responses in TypeScript — the patterns separating demo code from production code.
A support-ticket classifier works flawlessly in every manual test. Ships. Within a week, three separate problems surface: the LLM call is sitting directly in the request cycle, so a slow provider response means a slow page load for the support agent. A regex tries to extract a category from the model’s free-text response and breaks the day the model phrases its answer slightly differently. And nobody notices for two weeks that the same ticket, submitted twice by an impatient customer, gets classified — and billed — twice, because nothing cached or deduplicated the call.
None of these are AI problems. They’re the same category of mistake developers stopped making with database queries and third-party APIs years ago, reappearing because “call an LLM” still feels novel enough that the old discipline doesn’t automatically get applied to it. This post is seven patterns that close that gap — built on Prism PHP, the Laravel-native abstraction layer that’s become the standard way to talk to OpenAI, Anthropic, and other providers from Laravel in 2026 — covering the difference between code that works in a demo and code that survives a provider outage, a cost audit, and a frontend team that needs to actually type against the response.
1. Queue Every AI Call That Isn’t Rendering a Streamed Response
The single most common mistake: an LLM call sitting synchronously inside a controller action, blocking the request-response cycle on however long the provider takes to respond — which is not fast, not predictable, and occasionally not bounded at all if the provider is having a bad day.
// ❌ Blocks the request. A slow provider response is now a slow page load.
public function store(Request $request): RedirectResponse
{
$summary = Prism::text()
->using(Provider::OpenAI, 'gpt-5')
->withPrompt("Summarize this ticket: {$request->body}")
->asText();
Ticket::create(['body' => $request->body, 'summary' => $summary->text]);
return redirect()->route('tickets.index');
}
// ✅ Queued — the request returns immediately, the summary arrives moments later
public function store(Request $request): RedirectResponse
{
$ticket = Ticket::create(['body' => $request->body]);
SummarizeTicket::dispatch($ticket);
return redirect()->route('tickets.show', $ticket);
}
class SummarizeTicket implements ShouldQueue
{
public $tries = 3;
public $backoff = [10, 30, 90];
public function __construct(private Ticket $ticket) {}
public function handle(): void
{
$response = Prism::text()
->using(Provider::OpenAI, 'gpt-5')
->withPrompt("Summarize this ticket: {$this->ticket->body}")
->asText();
$this->ticket->update(['summary' => $response->text]);
broadcast(new TicketSummarized($this->ticket))->toOthers();
}
}
The $tries and $backoff here aren’t boilerplate — they’re the exact retry discipline Laravel’s queue system already provides for any unreliable external call, applied to an LLM call the same way it’d be applied to any other third-party API prone to transient failures. Broadcasting the result once the job completes (Reverb, in a real-time-capable app) is what lets the UI update the moment the summary is ready, without the user ever staring at a blocked page waiting for a provider round trip. The only legitimate exception to “queue it” is a response streamed directly to the user in real time — a chat interface, where the entire value is watching tokens arrive live, which is pattern three below.
2. Structured Output, Never Regex Against Free Text
Parsing a category, a sentiment score, or a boolean out of an LLM’s free-text response with a regex is fragile in a specific, predictable way: it works until the model’s phrasing shifts slightly — a version update, a prompt tweak, sometimes just normal model variance — and then it silently breaks or silently returns something subtly wrong, because the regex still “matches,” just the wrong substring.
// ❌ Parsing free text — breaks the day the model phrases things differently
$response = Prism::text()
->using(Provider::OpenAI, 'gpt-5')
->withPrompt("Classify this ticket's urgency as low, medium, or high: {$body}")
->asText();
preg_match('/\b(low|medium|high)\b/i', $response->text, $matches);
$urgency = $matches[1] ?? 'medium'; // silent, wrong fallback if the regex misses
// ✅ Structured output — the model's response is constrained to match a schema
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
$schema = new ObjectSchema(
name: 'ticket_classification',
description: 'Urgency and category classification for a support ticket',
properties: [
new StringSchema('urgency', 'One of: low, medium, high'),
new StringSchema('category', 'One of: billing, technical, account, other'),
],
requiredFields: ['urgency', 'category'],
);
$response = Prism::structured()
->using(Provider::OpenAI, 'gpt-5')
->withSchema($schema)
->withPrompt("Classify this ticket: {$body}")
->asStructured();
$urgency = $response->structured['urgency']; // guaranteed to exist, guaranteed shape
The difference isn’t cosmetic — a schema-constrained response is structurally guaranteed to conform, rather than relying on the model to voluntarily produce parseable text because it was asked nicely. This is the same principle covered in prior posts on structured LLM output for other languages: anywhere an LLM’s response feeds into code rather than a person’s eyes, the response needs to be constrained by a schema, not extracted by a pattern match against prose.
Even with a schema, validate the values, not just the shape. A schema guarantees urgency is a string; it doesn’t guarantee the model didn’t return "urgent" instead of "high" on an off day. Cast the structured response through a strict enum or a Laravel Data object before it touches business logic.
enum Urgency: string
{
case Low = 'low';
case Medium = 'medium';
case High = 'high';
}
$urgency = Urgency::tryFrom($response->structured['urgency']) ?? Urgency::Medium;
3. Streaming for Anything the User Is Actively Watching
The one case where the LLM call belongs directly in the request cycle: a chat interface, or anything else where the entire UX value is watching the response arrive token by token rather than waiting for the whole thing at once.
// A streamed chat endpoint — SSE, not a blocking JSON response
public function stream(Request $request): StreamedResponse
{
return response()->stream(function () use ($request) {
$stream = Prism::text()
->using(Provider::Anthropic, 'claude-sonnet-4-6')
->withPrompt($request->input('message'))
->asStream();
foreach ($stream as $chunk) {
echo "data: " . json_encode(['text' => $chunk->text]) . "\n\n";
ob_flush();
flush();
}
echo "data: [DONE]\n\n";
ob_flush();
flush();
}, 200, [
'Content-Type' => 'text/event-stream',
'Cache-Control' => 'no-cache',
'X-Accel-Buffering' => 'no', // required behind nginx, or chunks get buffered anyway
]);
}
X-Accel-Buffering: no is the header most tutorials skip and the one that causes “streaming works locally, arrives as one giant chunk in production” — nginx buffers proxied responses by default, and without this header explicitly disabling that behavior, every chunk sits in a buffer until the full response completes, defeating the entire point of streaming despite the PHP code being completely correct.
// Frontend — consuming the stream token by token
const eventSource = new EventSource('/api/chat/stream?' + new URLSearchParams({ message }))
eventSource.onmessage = (event) => {
if (event.data === '[DONE]') {
eventSource.close()
return
}
const { text } = JSON.parse(event.data)
chatMessage.value += text
}
The rule that separates this pattern from pattern one, cleanly: if a person is actively watching the response render in real time, stream it, synchronously, in the request. If the response is consumed later — an email, a dashboard, a background classification — queue it. Conflating the two is how a chat feature ends up needlessly queued and delayed, or how a background summarization job ends up needlessly blocking a page load.
4. Retries and Fallback Models, Not a Single Point of Failure
An LLM provider having a slow or degraded moment is a “when,” not an “if” — and code that treats a single provider, single model, no-retry call as reliable infrastructure will eventually fail in front of a user for a reason that had nothing to do with the application itself.
// A resilient wrapper — retry, then fall back to a different provider entirely
class ResilientCompletion
{
public function complete(string $prompt): string
{
return retry(2, function () use ($prompt) {
return Prism::text()
->using(Provider::OpenAI, 'gpt-5')
->withPrompt($prompt)
->asText()
->text;
}, 200) ?? $this->fallback($prompt);
}
private function fallback(string $prompt): string
{
try {
return Prism::text()
->using(Provider::Anthropic, 'claude-sonnet-4-6')
->withPrompt($prompt)
->asText()
->text;
} catch (\Throwable $e) {
Log::error('Both primary and fallback AI providers failed', ['error' => $e->getMessage()]);
throw new AiUnavailableException();
}
}
}
Laravel’s own retry() helper handles the transient-failure case — a momentary timeout, a rate limit that clears in 200ms — cheaply, with no new dependency. The fallback to a second provider is for the less common but real case where the primary provider is degraded for longer than a couple of quick retries can paper over, and it’s the difference between an AI-dependent feature going fully dark during a provider incident and quietly degrading to a different model instead. This is worth building once, as a shared wrapper, rather than re-implemented per feature — the same discipline as the shared prompt-template pattern from prompt engineering: a resilience pattern re-derived from scratch in every controller that calls an LLM is a resilience pattern that’s inconsistently applied, and inconsistent is functionally the same as absent the one time it actually matters.
5. Cache Identical Requests — the Cost Control Nobody Adds Until the Bill Arrives
The support-ticket-classified-twice problem from the opening is a specific case of a general one: identical or near-identical AI requests, made more than once, cost more than once, with no functional benefit to paying twice for the same answer.
class TicketClassifier
{
public function classify(string $body): array
{
$cacheKey = 'ticket-classification:' . hash('sha256', trim(mb_strtolower($body)));
return Cache::remember($cacheKey, now()->addDays(7), function () use ($body) {
$response = Prism::structured()
->using(Provider::OpenAI, 'gpt-5')
->withSchema($this->schema())
->withPrompt("Classify: {$body}")
->asStructured();
return $response->structured;
});
}
}
This is the same normalized-hash caching pattern covered in the semantic-search post, applied generally: hash the normalized input, cache the structured result, and a duplicate submission — a double-click, a retried form, an actually-identical ticket from a different customer — costs one API call instead of one per occurrence. The TTL is a real decision, not a default to leave alone: content that can legitimately change its correct classification over time (sentiment on a live-updating thread) needs a much shorter TTL, or no caching at all, than a static classification of a fixed, immutable piece of text.
Idempotency matters here too, the same way it matters for a queued job. If SummarizeTicket from pattern one gets dispatched twice for the same ticket — a queue redelivery, a duplicate form submission that both got queued before the cache check — the job itself should check whether a summary already exists before making the call, not rely solely on the cache layer to catch every duplicate path into the same work.
6. Typed AI Responses, Shared to TypeScript — Not Re-Guessed on the Frontend
The gap that causes the most frontend bugs in an AI feature: the backend’s structured response shape and the frontend’s TypeScript type for it, maintained independently, drifting the moment one changes without the other — exactly the shared-type problem covered for Eloquent models in the Laravel + Vue architecture post, except here the “backend model” is an LLM’s structured output, which is if anything more likely to drift, because a schema change might come from a prompt tweak rather than a deliberate migration.
// app/Data/TicketClassification.php — Laravel Data object, the actual contract
class TicketClassification extends Data
{
public function __construct(
public Urgency $urgency,
public string $category,
public string $summary,
) {}
}
// The controller returns the Data object — Inertia/API response is now
// backed by a real, typed PHP class, not a loose array
public function show(Ticket $ticket): Response
{
return Inertia::render('Tickets/Show', [
'ticket' => $ticket,
'classification' => TicketClassification::from($ticket->classification_data),
]);
}
// resources/js/types/models.ts — generated from the PHP Data class,
// not hand-typed and left to drift
export interface TicketClassification {
urgency: 'low' | 'medium' | 'high'
category: string
summary: string
}
Generating this TypeScript interface from the actual PHP Data class — via spatie/laravel-typescript-transformer or an equivalent, run as part of the build rather than a manual step — closes the drift gap the same way it does for ordinary Eloquent models. The AI-specific reason this matters more here than for a typical model: a TicketClassification‘s shape is defined by a schema that lives inside a prompt-adjacent PHP file, edited by whoever’s iterating on prompt quality that week, who may have no reason to think of themselves as making a “breaking API change” — and without generated types keeping the frontend honest, they aren’t wrong to not think of it that way, right up until a component destructures a field that silently stopped being sent.
7. Never Trust Structured Output for an Action Without a Validation Boundary
The pattern that separates a genuinely production-ready AI integration from one that merely looks production-ready: treating an LLM’s output — structured, schema-validated, well-tested — as input to validate, not as a trusted command to execute directly, the moment that output triggers a real side effect rather than just being displayed.
// ❌ The model's structured output directly triggers a real action —
// no validation boundary between "the AI suggested this" and "this happened"
$response = Prism::structured()
->using(Provider::OpenAI, 'gpt-5')
->withSchema($refundSchema)
->withPrompt("Determine the appropriate refund for this complaint: {$complaint}")
->asStructured();
Refund::create([
'order_id' => $response->structured['order_id'],
'amount' => $response->structured['amount'], // trusting the model's math and judgment directly
]);
// ✅ The model's output is a recommendation, validated against real
// constraints before anything actually happens
$response = Prism::structured()
->using(Provider::OpenAI, 'gpt-5')
->withSchema($refundSchema)
->withPrompt("Determine the appropriate refund for this complaint: {$complaint}")
->asStructured();
$order = Order::findOrFail($response->structured['order_id']);
$suggestedAmount = $response->structured['amount'];
// Real business constraints, enforced in code, regardless of what the model said
abort_if($suggestedAmount > $order->total, 422, 'Suggested refund exceeds order total.');
abort_if($order->refunds()->sum('amount') + $suggestedAmount > $order->total, 422, 'Refund would exceed order total.');
RefundRequest::create([
'order_id' => $order->id,
'suggested_amount' => $suggestedAmount,
'status' => 'pending_approval', // a human confirms before money actually moves,
// for any action above a deliberately low
// auto-approval threshold
]);
An LLM producing schema-valid, well-formatted output is not the same claim as the LLM being correct about the specific numbers or decisions inside that output — a model can confidently return a refund amount that exceeds the order total, a category that doesn’t exist in the actual business taxonomy, or a confidently-wrong read of an ambiguous complaint. The validation boundary here is doing exactly the job a Form Request does for user input: never assume a value is safe to act on just because it arrived in the expected shape. For anything where the action is expensive, irreversible, or touches real money, a human-in-the-loop approval step for AI-suggested actions above a defined threshold isn’t excessive caution — it’s the same proportional-risk judgment applied to force-deleting a row with foreign key dependents, just with the LLM’s confidence standing in for the assumption that used to be “the code we wrote must be right.”
The Complete Shape
1. Queue it — unless it's rendered live to a watching user, dispatch it
2. Structure it — schema-constrained output, never regex against prose
3. Stream it — only for the in-request, actively-watched case
4. Retry + fall back — a resilient wrapper, built once, reused everywhere
5. Cache it — normalized-hash cache on identical requests, real TTL decision
6. Type it — generated TypeScript from the actual PHP Data contract
7. Validate it — an AI recommendation is input to check, not a command to trust
The One Rule
Every pattern here is the same lesson, applied to a technology that still feels new enough for developers to forget they already know it: an LLM call is an external, unreliable, costly dependency, exactly like any third-party API — and every discipline that discipline already demands (queue what doesn’t need to block, retry what’s transient, cache what’s redundant, validate what triggers an action, type what crosses a language boundary) applies to it without needing to be reinvented from scratch just because the response happens to come from a model instead of a REST endpoint. The gap between demo code and production code was never really about AI. It’s the same gap it’s always been — code that assumes the happy path is the only path, versus code that was built for the day it isn’t.