Production AI Agents in Laravel: Streaming, Token Budgets, and Structured Output Contracts

The Gap Between Demo and Production AI Agents

Most Laravel AI tutorials stop at $client->chat(). Production agents need three things demos skip: streaming responses that don't time out, hard token budgets that protect your bill, and structured output contracts that fail loudly when a model returns garbage.

This article tackles all three with concrete, opinionated patterns.

Streaming with Server-Sent Events

Laravel's StreamedResponse is the right primitive. Pair it with a generator-based OpenAI client call and you get true SSE without a WebSocket server.

// routes/api.php
Route::post('/agent/stream', AgentStreamController::class);

// app/Http/Controllers/AgentStreamController.php
final class AgentStreamController
{
    public function __invoke(AgentRequest $request, AgentService $agent): StreamedResponse
    {
        return response()->stream(
            function () use ($request, $agent): void {
                foreach ($agent->stream($request->validated('prompt')) as $chunk) {
                    echo "data: " . json_encode(['text' => $chunk]) . "\n\n";
                    ob_flush();
                    flush();
                }
                echo "data: [DONE]\n\n";
            },
            headers: ['Content-Type' => 'text/event-stream', 'X-Accel-Buffering' => 'no']
        );
    }
}

The X-Accel-Buffering: no header is critical when Nginx sits in front — without it, Nginx buffers the entire response.

Enforcing Token Budgets at the Application Layer

Don't rely solely on max_tokens in the API call. A multi-turn agent accumulates context silently. Track token usage yourself and abort before you hit a cost cliff.

final class TokenBudget
{
    private int $used = 0;

    public function __construct(private readonly int $limit) {}

    public function consume(int $tokens): void
    {
        $this->used += $tokens;
        if ($this->used > $this->limit) {
            throw new TokenBudgetExceededException(
                "Budget of {$this->limit} tokens exceeded (used: {$this->used})"
            );
        }
    }

    public function remaining(): int
    {
        return max(0, $this->limit - $this->used);
    }
}

Inject a TokenBudget into your agent loop and call consume() after each API response using the usage object the API returns. This gives you per-request, per-user, or per-tenant budget enforcement — whichever granularity your SaaS needs.

$budget = new TokenBudget(limit: 8_000);

foreach ($turns as $turn) {
    $response = $this->client->chat($turn->toMessages(), maxTokens: $budget->remaining());
    $budget->consume($response->usage->totalTokens);
    // ...
}

Structured Output Contracts with Readonly DTOs

JSON mode is not a contract. Models hallucinate keys, change nesting, or return null where you expect a string. Validate every structured response against a typed DTO immediately after deserialization.

readonly class SentimentResult
{
    public function __construct(
        public readonly string $sentiment,  // 'positive'|'negative'|'neutral'
        public readonly float  $confidence, // 0.0–1.0
        public readonly string $summary,
    ) {}

    public static function fromArray(array $data): self
    {
        $validated = validator($data, [
            'sentiment'  => ['required', 'string', Rule::in(['positive','negative','neutral'])],
            'confidence' => ['required', 'numeric', 'min:0', 'max:1'],
            'summary'    => ['required', 'string', 'max:500'],
        ])->validate();

        return new self(...$validated);
    }
}

Call SentimentResult::fromArray(json_decode($response->content, true)) and let Laravel's validator throw a ValidationException on malformed output. This surfaces model regressions immediately rather than letting bad data propagate into your database.

Prompt-Side Contract Enforcement

Include the schema in your system prompt as a JSON Schema snippet. Models with JSON mode enabled respect it more reliably than natural-language instructions alone. Store the schema alongside the DTO so they stay in sync:

public static function jsonSchema(): array
{
    return [
        'type' => 'object',
        'required' => ['sentiment', 'confidence', 'summary'],
        'properties' => [
            'sentiment'  => ['type' => 'string', 'enum' => ['positive','negative','neutral']],
            'confidence' => ['type' => 'number', 'minimum' => 0, 'maximum' => 1],
            'summary'    => ['type' => 'string', 'maxLength' => 500],
        ],
    ];
}

Key Takeaways

• Use StreamedResponse with X-Accel-Buffering: no for true SSE through Nginx.
• Track token usage from API responses; enforce budgets in application code, not just max_tokens.
• Validate every structured model response against a typed readonly DTO using Laravel's validator.
• Co-locate the JSON Schema with the DTO so prompt and validation contracts never drift.
• Throw typed exceptions (TokenBudgetExceededException, ValidationException) so callers can handle failures explicitly.