Scoped context, streaming responses, user-specific memory, cost controls, fallback behaviour when the API is down, and the one architectural decision that kept AI completely isolated from the rest of the codebase — a practical implementation story, not a toy demo.
The demo version of “add AI to your Laravel app” takes about 15 minutes. You call the API, you stream the response, you render it in a chat bubble. It works in a video. Then you actually think about the production version: what context does the AI see? Who can use it and how often? What happens when Anthropic’s API is down at 2am? Where does this code live so it doesn’t contaminate the rest of the application? What does a 50,000-token conversation history cost per user per month?
Those are the questions nobody covers. This post is about those questions, working from the implementation I shipped into a multi-tenant SaaS project management app — a copilot that knows about the user’s current project, can summarise tasks, suggest priorities, and answer questions about the team’s work. Four hours to go from nothing to deployed. Here’s exactly how.
The Architectural Decision That Made Everything Else Easier
Before any code: the copilot is a feature, not a service. That’s the framing that kept it isolated.
Every Laravel developer knows the instinct — create a CopilotService, inject it into controllers, call it from wherever it’s needed. That works until the copilot needs project context, user history, cost tracking, and fallback logic. Then the service grows, it gets injected in more places, and the AI layer starts leaking into the rest of the application.
The decision that prevented this: the copilot gets its own controller, its own routes, its own middleware group, and its own directory. Nothing else in the application calls the copilot. The copilot calls into the rest of the application — reads projects, reads tasks, reads users — but the dependency only runs one way.
app/
├── Http/
│ └── Controllers/
│ └── Copilot/
│ ├── CopilotController.php ← handles HTTP, streams response
│ └── CopilotController.php
├── AI/
│ ├── Agents/
│ │ └── ProjectCopilot.php ← the agent class
│ ├── Context/
│ │ └── ProjectContextBuilder.php ← gathers project data
│ └── Memory/
│ └── ConversationMemory.php ← per-user, per-project history
Every AI concern lives in app/AI/. The rest of the application doesn’t know it exists.
Installing the Laravel AI SDK
composer require laravel/ai
php artisan ai:install
ai:install publishes config/ai.php and adds the AI_PROVIDER and AI_KEY variables to .env. The config file is where provider selection, model defaults, and cost controls live — not in service classes or environment-specific logic scattered through the codebase.
// config/ai.php
return [
'default' => env('AI_PROVIDER', 'anthropic'),
'providers' => [
'anthropic' => [
'api_key' => env('ANTHROPIC_API_KEY'),
'model' => env('AI_MODEL', 'claude-sonnet-4-6'),
],
'openai' => [
'api_key' => env('OPENAI_API_KEY'),
'model' => env('AI_MODEL', 'gpt-4o'),
],
],
'limits' => [
'max_tokens' => 4096,
'context_tokens' => 8192, // max tokens in conversation history
'daily_per_user' => 50, // max requests per user per day
],
];
The limits block isn’t part of the SDK — it’s application configuration I added. The SDK enforces max_tokens; the daily_per_user and context_tokens limits are enforced in middleware and the context builder respectively.
The Agent Class
The Laravel AI SDK uses agent classes to encapsulate model behaviour. The ProjectCopilot agent defines the instructions, the context it receives, and how it responds:
<?php
namespace App\AI\Agents;
use App\Models\Project;
use App\Models\User;
use Illuminate\Support\Stringable;
use Laravel\Ai\Contracts\Agent;
use Laravel\Ai\Promptable;
class ProjectCopilot implements Agent
{
use Promptable;
public function __construct(
private readonly Project $project,
private readonly User $user,
private readonly string $context,
) {}
public function instructions(): Stringable|string
{
return <<<INSTRUCTIONS
You are a helpful project management assistant for the project "{$this->project->name}".
You have access to the following information about this project:
{$this->context}
Guidelines:
- Answer questions specifically about this project and its tasks
- Do not discuss other projects the user may have access to
- Do not make up information not provided in the context
- If asked about something outside the project context, say so clearly
- Keep responses concise — this is a sidebar assistant, not a document
- Today's date is {{ now()->toFormattedDateString() }}
The user you are helping is {$this->user->name}.
INSTRUCTIONS;
}
}
The instructions() method injects the project name, the built context, and the user’s name. The scoping rules in the guidelines — “do not discuss other projects” — are the prompt-level equivalent of tenant scoping in Eloquent. They’re not the only protection (the context builder is the real guard), but they reinforce the intent.
The Context Builder — The Most Important Class in the System
The context builder decides what the AI sees. It’s also where multi-tenancy lives for the AI layer. An AI that gets context it shouldn’t — another tenant’s tasks, another user’s private notes — is a data breach, not a bug.
<?php
namespace App\AI\Context;
use App\Models\Project;
use App\Models\User;
class ProjectContextBuilder
{
private const MAX_TASKS = 50;
private const MAX_COMMENTS = 20;
private const CONTEXT_LIMIT = 6000; // characters, not tokens
public function build(Project $project, User $user): string
{
// Every query is scoped to the project AND verified against the user's access
// If the user doesn't belong to this project, abort() is called by the middleware
// before this runs — but we scope anyway as a defense-in-depth measure
$tasks = $project->tasks()
->with(['assignee:id,name', 'status'])
->orderByDesc('updated_at')
->limit(self::MAX_TASKS)
->get()
->map(fn ($task) => [
'title' => $task->title,
'status' => $task->status->name,
'assignee' => $task->assignee?->name ?? 'Unassigned',
'due' => $task->due_date?->format('M d'),
'priority' => $task->priority,
]);
$recentActivity = $project->activities()
->with('user:id,name')
->latest()
->limit(self::MAX_COMMENTS)
->get()
->map(fn ($activity) => "{$activity->user->name}: {$activity->description}");
$context = collect([
"PROJECT: {$project->name}",
"STATUS: {$project->status}",
"DUE DATE: {$project->due_date?->format('M d, Y') ?? 'Not set'}",
"",
"TASKS ({$tasks->count()}):",
...$tasks->map(fn ($t) =>
"- [{$t['status']}] {$t['title']} (Assigned: {$t['assignee']}, Due: {$t['due'] ?? 'N/A'}, Priority: {$t['priority']})"
),
"",
"RECENT ACTIVITY:",
...$recentActivity,
])->join("\n");
// Truncate to stay within context limits
// Truncating from the end of activity (least important) not the beginning
return substr($context, 0, self::CONTEXT_LIMIT);
}
}
The MAX_TASKS = 50 and CONTEXT_LIMIT = 6000 constants are the cost controls at the context level. The AI only sees the 50 most recently updated tasks and the last 20 activity entries. A project with 500 tasks doesn’t produce a 500-task context that costs $2 per query.
Per-User Conversation Memory
A copilot without memory is a chat bot. Every message is a fresh start. The user has to re-explain what they were asking about on every turn.
Conversation history is stored in the database, scoped to the user and the project:
// Migration
Schema::create('copilot_conversations', function (Blueprint $table) {
$table->id();
$table->foreignId('user_id')->constrained()->cascadeOnDelete();
$table->foreignId('project_id')->constrained()->cascadeOnDelete();
$table->json('messages'); // [{role, content}, ...]
$table->unsignedInteger('token_count')->default(0);
$table->timestamps();
$table->unique(['user_id', 'project_id']); // one conversation per user per project
$table->index(['user_id', 'project_id']);
});
<?php
namespace App\AI\Memory;
use App\Models\CopilotConversation;
use App\Models\Project;
use App\Models\User;
class ConversationMemory
{
private const MAX_HISTORY_TOKENS = 4000;
private const TOKENS_PER_CHAR = 4; // rough approximation
public function get(User $user, Project $project): array
{
$conversation = CopilotConversation::where('user_id', $user->id)
->where('project_id', $project->id)
->first();
return $conversation?->messages ?? [];
}
public function append(User $user, Project $project, string $role, string $content): void
{
$conversation = CopilotConversation::firstOrCreate(
['user_id' => $user->id, 'project_id' => $project->id],
['messages' => [], 'token_count' => 0],
);
$messages = $conversation->messages;
$messages[] = ['role' => $role, 'content' => $content];
// Trim history to stay within token budget
// Drop the oldest messages (but never the first system context)
$messages = $this->trimToLimit($messages);
$conversation->update([
'messages' => $messages,
'token_count' => $this->estimateTokens($messages),
]);
}
public function clear(User $user, Project $project): void
{
CopilotConversation::where('user_id', $user->id)
->where('project_id', $project->id)
->delete();
}
private function trimToLimit(array $messages): array
{
$estimatedTokens = $this->estimateTokens($messages);
// Drop oldest messages from the front until we're within budget
while ($estimatedTokens > self::MAX_HISTORY_TOKENS && count($messages) > 1) {
array_shift($messages);
$estimatedTokens = $this->estimateTokens($messages);
}
return $messages;
}
private function estimateTokens(array $messages): int
{
$charCount = collect($messages)->sum(fn ($m) => strlen($m['content']));
return (int) ceil($charCount / self::TOKENS_PER_CHAR);
}
}
The MAX_HISTORY_TOKENS = 4000 limit means no conversation grows unboundedly. When the history exceeds the budget, oldest messages are dropped from the front. The user’s most recent exchanges are preserved. The token_count column is stored for cost reporting without making a separate API call to count tokens.
The Controller — Streaming the Response
The controller ties everything together. It validates the request, checks the daily limit, builds the context, retrieves memory, prompts the agent, and streams the response back:
<?php
namespace App\Http\Controllers\Copilot;
use App\AI\Agents\ProjectCopilot;
use App\AI\Context\ProjectContextBuilder;
use App\AI\Memory\ConversationMemory;
use App\Models\Project;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\RateLimiter;
use Symfony\Component\HttpFoundation\StreamedResponse;
class CopilotController extends Controller
{
public function __construct(
private readonly ProjectContextBuilder $contextBuilder,
private readonly ConversationMemory $memory,
) {}
public function ask(Request $request, Project $project): StreamedResponse
{
$request->validate([
'message' => ['required', 'string', 'max:2000'],
]);
// Daily rate limit — 50 requests per user per day
$key = "copilot:{$request->user()->id}:daily";
$limit = config('ai.limits.daily_per_user', 50);
if (RateLimiter::tooManyAttempts($key, $limit)) {
$seconds = RateLimiter::availableIn($key);
abort(429, "Daily AI limit reached. Resets in {$seconds} seconds.");
}
RateLimiter::hit($key, 86400); // decay in 24 hours
$user = $request->user();
$message = $request->input('message');
// Build project context — scoped, truncated, cost-controlled
$context = $this->contextBuilder->build($project, $user);
// Retrieve conversation history for continuity
$history = $this->memory->get($user, $project);
// Persist the user's message before streaming starts
// so it's saved even if the stream fails mid-way
$this->memory->append($user, $project, 'user', $message);
return response()->stream(function () use ($project, $user, $context, $history, $message) {
$agent = new ProjectCopilot($project, $user, $context);
$fullResponse = '';
try {
$stream = $agent
->withHistory($history)
->stream($message);
foreach ($stream as $chunk) {
$token = $chunk->text ?? '';
if ($token) {
$fullResponse .= $token;
echo "data: " . json_encode(['token' => $token]) . "\n\n";
ob_flush();
flush();
}
}
// Stream complete — persist assistant response
$this->memory->append($user, $project, 'assistant', $fullResponse);
echo "data: [DONE]\n\n";
ob_flush();
flush();
} catch (\Exception $e) {
// Graceful degradation — send error event, don't crash the stream
echo "data: " . json_encode(['error' => $this->userFacingError($e)]) . "\n\n";
ob_flush();
flush();
report($e); // log to your error tracker
}
}, 200, [
'Content-Type' => 'text/event-stream',
'Cache-Control' => 'no-cache',
'X-Accel-Buffering' => 'no', // disables Nginx buffering
]);
}
public function clearHistory(Request $request, Project $project): \Illuminate\Http\JsonResponse
{
$this->memory->clear($request->user(), $project);
return response()->json(['cleared' => true]);
}
private function userFacingError(\Exception $e): string
{
return match (true) {
str_contains($e->getMessage(), 'overloaded') => 'The AI is busy right now. Try again in a moment.',
str_contains($e->getMessage(), 'rate_limit') => 'Rate limit reached. Try again shortly.',
str_contains($e->getMessage(), 'timeout') => 'The request timed out. Try a shorter question.',
default => 'Something went wrong. The AI is unavailable right now.',
};
}
}
The X-Accel-Buffering: no header is the Nginx detail most streaming implementations miss. Without it, Nginx buffers the entire response before sending it to the client. The stream appears to hang until the full response is generated, then it all arrives at once. The header disables that buffering.
The Middleware — Authorization and Project Scoping
Two middleware classes protect the copilot routes. One verifies the user belongs to the project. One verifies the copilot feature is enabled for the tenant.
// app/Http/Middleware/AuthorizeCopilotAccess.php
class AuthorizeCopilotAccess
{
public function handle(Request $request, Closure $next): Response
{
$project = $request->route('project');
// Verify the authenticated user belongs to this project
if (! $request->user()->projects()->where('projects.id', $project->id)->exists()) {
abort(403, 'You do not have access to this project.');
}
// Verify the tenant's plan includes the copilot feature
if (! $request->user()->tenant->hasFeature('copilot')) {
abort(403, 'Upgrade your plan to access the AI copilot.');
}
return $next($request);
}
}
// routes/api.php
Route::middleware(['auth:sanctum', 'throttle:api'])
->prefix('api')
->group(function () {
Route::middleware(AuthorizeCopilotAccess::class)
->prefix('projects/{project}/copilot')
->group(function () {
Route::post('ask', [CopilotController::class, 'ask']);
Route::delete('history', [CopilotController::class, 'clearHistory']);
});
});
The route structure — projects/{project}/copilot/ask — means the project is always in the URL and always available to the middleware. There’s no way to call the copilot without specifying which project you’re talking about.
Fallback Behaviour When the API Is Down
The userFacingError() method in the controller handles runtime errors during streaming. But the more important fallback is at the application level: what does the UI show when the copilot endpoint returns an error?
The Vue component that drives the copilot panel handles three states:
// Simplified state machine for the copilot UI
type CopilotState =
| { status: 'idle' }
| { status: 'streaming'; partial: string }
| { status: 'complete'; response: string }
| { status: 'rate_limited'; resetsAt: string }
| { status: 'unavailable'; message: string } // API down, timeout, etc.
// When the stream returns an error event
if (event.error) {
state.value = {
status: 'unavailable',
message: event.error,
}
return
}
The unavailable state renders a message in the copilot panel — “The AI assistant is temporarily unavailable. The rest of the app is working normally.” The critical detail: the rest of the application is completely unaffected. The project view, the task board, the activity feed — they don’t know the copilot exists. Because of the isolation decision made at the start, the copilot failing has exactly zero impact on anything else.
Cost Controls in Summary
The full cost control stack, across all layers:
Layer 1 — Rate limiting (middleware + RateLimiter)
→ 50 requests per user per day
→ Laravel's RateLimiter with 24-hour decay
Layer 2 — Context truncation (ProjectContextBuilder)
→ Maximum 50 tasks in context
→ Maximum 20 activity entries in context
→ Hard character limit (CONTEXT_LIMIT = 6000 chars)
Layer 3 — History trimming (ConversationMemory)
→ Maximum 4000 estimated tokens of conversation history
→ Oldest messages dropped when limit is exceeded
Layer 4 — Response token limit (config/ai.php)
→ max_tokens: 4096
→ The SDK passes this to the provider — responses can't exceed it
Layer 5 — Model selection (config/ai.php)
→ claude-sonnet-4-6 (not Opus) for the default model
→ AI_MODEL in .env allows changing per environment without code changes
At 50 requests per user per day with an average context of ~2000 tokens and average response of ~500 tokens, the per-user cost at claude-sonnet-4-6 pricing is roughly $0.10–0.20 per day of heavy use. For a SaaS with per-seat pricing, that’s a known, bounded cost per seat.
The 4-Hour Breakdown
What actually took time, and what didn’t:
Hour 1 — Architecture and installation
→ Directory structure decision (30 min)
→ composer require laravel/ai, config setup (15 min)
→ Migration and model for conversation history (15 min)
Hour 2 — Agent and context builder
→ ProjectCopilot agent class (20 min)
→ ProjectContextBuilder with scoping and limits (40 min)
Hour 3 — Controller, middleware, and streaming
→ CopilotController with streaming and error handling (45 min)
→ AuthorizeCopilotAccess middleware (15 min)
Hour 4 — Vue component and integration testing
→ EventSource / fetch-based streaming in Vue (30 min)
→ Testing with real API, edge cases (20 min)
→ Cost control verification (10 min)
The Vue component is the only part not shown here — it’s covered in the Streaming AI in Vue post. The backend integration is complete as written.
What Would Have Taken Longer Without the SDK
Before the Laravel AI SDK shipped in February 2026, adding an AI copilot to a Laravel application meant: choosing a provider, installing their PHP SDK, writing a service class to wrap it, handling streaming manually with GuzzleHttp\RequestOptions::STREAM, normalising error formats across providers, and writing your own retry logic. Most implementations ended up as a fat service class that knew too much about both the AI layer and the application domain.
The SDK removed that scaffolding entirely. The agent class is the AI concern. The controller is the HTTP concern. The context builder is the application domain concern. Each one is small, testable, and replaceable. Swapping from Anthropic to OpenAI is changing AI_PROVIDER in .env — the agent class doesn’t change.
That’s the version of “add AI to your Laravel app” that survives contact with production.