Production AI Agents in Laravel: Streaming, Token Budgets, and Structured Output Contracts

The Gap Between Demo and Production

Most Laravel AI tutorials stop at Http::post('https://api.openai.com/v1/chat/completions', [...]). That works for a demo. It falls apart in production when a single runaway prompt burns your monthly token budget in an afternoon, a slow model response blocks a PHP-FPM worker for 45 seconds, or a hallucinated JSON blob crashes your downstream pipeline.

This article covers three concrete patterns that close that gap: streaming with SSE, token budget middleware, and structured output contracts with readonly DTOs.

1. Streaming Responses Over SSE

OpenAI's streaming API sends text/event-stream chunks. Laravel's StreamedResponse lets you forward them to the browser without buffering the entire completion in memory.

// routes/web.php
Route::get('/chat/stream', ChatStreamController::class);

// app/Http/Controllers/ChatStreamController.php
final class ChatStreamController
{
    public function __invoke(ChatRequest $request, AgentService $agent): StreamedResponse
    {
        return response()->stream(
            function () use ($request, $agent) {
                foreach ($agent->stream($request->validated('message')) as $chunk) {
                    echo "data: {$chunk}\n\n";
                    ob_flush();
                    flush();
                }
                echo "data: [DONE]\n\n";
            },
            200,
            [
                'Content-Type' => 'text/event-stream',
                'X-Accel-Buffering' => 'no', // critical for nginx
                'Cache-Control' => 'no-cache',
            ]
        );
    }
}

Inside AgentService::stream(), use a generator that reads the chunked HTTP response line by line:

public function stream(string $prompt): \Generator
{
    $response = Http::withToken(config('services.openai.key'))
        ->withOptions(['stream' => true])
        ->post('https://api.openai.com/v1/chat/completions', [
            'model' => 'gpt-4o',
            'stream' => true,
            'messages' => [['role' => 'user', 'content' => $prompt]],
        ]);

    $body = $response->toPsrResponse()->getBody();

    while (! $body->eof()) {
        $line = trim($body->read(512));
        if (str_starts_with($line, 'data: ') && $line !== 'data: [DONE]') {
            $data = json_decode(substr($line, 6), true);
            $content = $data['choices'][0]['delta']['content'] ?? '';
            if ($content !== '') {
                yield $content;
            }
        }
    }
}

Note: Set fastcgi_buffering off in nginx and ensure PHP-FPM output buffering is disabled for the relevant location block.

2. Token Budget Middleware

Token budgets belong at the pipeline layer, not scattered across controllers. A dedicated middleware class can inspect the request, estimate prompt tokens, and reject or downgrade the model before the HTTP call is made.

final class EnforceTokenBudget
{
    // Rough heuristic: 1 token ≈ 4 chars
    private const CHARS_PER_TOKEN = 4;
    private const HARD_LIMIT = 8_000;

    public function handle(AgentPayload $payload, Closure $next): mixed
    {
        $estimated = (int) ceil(
            strlen($payload->systemPrompt . $payload->userMessage) / self::CHARS_PER_TOKEN
        );

        if ($estimated > self::HARD_LIMIT) {
            throw new TokenBudgetExceededException($estimated, self::HARD_LIMIT);
        }

        // Downgrade model for large-but-acceptable prompts
        if ($estimated > 4_000) {
            $payload = $payload->withModel('gpt-4o-mini');
        }

        return $next($payload);
    }
}

Wire it into a custom AgentPipeline:

$result = app(Pipeline::class)
    ->send(new AgentPayload($system, $user))
    ->through([
        EnforceTokenBudget::class,
        SanitizeUserInput::class,
        InjectRagContext::class,
    ])
    ->thenReturn();

3. Structured Output Contracts

OpenAI's response_format with json_schema mode guarantees the model returns a specific shape. Pair it with a PHP 8.3 readonly DTO and a custom cast for end-to-end type safety.

readonly final class ProductSummary
{
    public function __construct(
        public string $name,
        public string $oneLinePitch,
        /** @var list<string> */
        public array $keyFeatures,
        public int $estimatedPriceUsd,
    ) {}

    public static function fromArray(array $data): self
    {
        return new self(
            name: $data['name'],
            oneLinePitch: $data['one_line_pitch'],
            keyFeatures: $data['key_features'],
            estimatedPriceUsd: $data['estimated_price_usd'],
        );
    }
}

Pass the schema to OpenAI:

$response = Http::withToken(config('services.openai.key'))
    ->post('https://api.openai.com/v1/chat/completions', [
        'model' => 'gpt-4o',
        'response_format' => [
            'type' => 'json_schema',
            'json_schema' => [
                'name' => 'product_summary',
                'strict' => true,
                'schema' => [
                    'type' => 'object',
                    'properties' => [
                        'name' => ['type' => 'string'],
                        'one_line_pitch' => ['type' => 'string'],
                        'key_features' => ['type' => 'array', 'items' => ['type' => 'string']],
                        'estimated_price_usd' => ['type' => 'integer'],
                    ],
                    'required' => ['name', 'one_line_pitch', 'key_features', 'estimated_price_usd'],
                    'additionalProperties' => false,
                ],
            ],
        ],
        'messages' => [['role' => 'user', 'content' => $prompt]],
    ]);

$summary = ProductSummary::fromArray(
    json_decode($response->json('choices.0.message.content'), true)
);

Because strict: true is set, OpenAI will never return extra keys or omit required ones. Your fromArray factory becomes a safe assertion, not defensive guesswork.

Key Takeaways

• Use response()->stream() with a generator and X-Accel-Buffering: no for true SSE streaming in Laravel.
• Enforce token budgets in a pipeline middleware layer, not inside controllers or service methods.
• json_schema with strict: true eliminates hallucinated keys; map the response to a readonly DTO immediately.
• Estimate prompt tokens early (chars ÷ 4 is good enough for budget checks) to avoid surprise costs.
• Model downgrade logic belongs in the pipeline, keeping your agent service stateless and testable.