Production AI Agents in Laravel: Streaming, Token Budgets, and Structured Output Contracts

The Problem With Naive AI Integration

Most Laravel + LLM tutorials stop at Http::post('https://api.openai.com/v1/chat/completions', [...]) and call it done. That works for demos. In production you need three things the tutorials skip:

1. Streaming — users shouldn't stare at a spinner for 8 seconds.
2. Token budgets — unbounded prompts destroy your billing and latency SLAs.
3. Structured output contracts — raw JSON strings from an LLM are not domain objects.

Let's solve all three without a heavy third-party SDK.

Streaming Responses to the Browser

OpenAI's stream: true returns server-sent events. Laravel's StreamedResponse pipes them straight to the client.

// app/Http/Controllers/AgentController.php
public function stream(Request $request): StreamedResponse
{
    $prompt = $request->validated()['prompt'];

    return response()->stream(function () use ($prompt) {
        $stream = Http::withToken(config('services.openai.key'))
            ->withOptions(['stream' => true])
            ->post('https://api.openai.com/v1/chat/completions', [
                'model'    => 'gpt-4o-mini',
                'stream'   => true,
                'messages' => [['role' => 'user', 'content' => $prompt]],
            ])->toPsrResponse()->getBody();

        while (! $stream->eof()) {
            $line = trim($stream->read(512));
            if (str_starts_with($line, 'data: ')) {
                $payload = substr($line, 6);
                if ($payload === '[DONE]') break;
                $delta = json_decode($payload, true)['choices'][0]['delta']['content'] ?? '';
                echo "data: {$delta}\n\n";
                ob_flush(); flush();
            }
        }
    }, 200, ['Content-Type' => 'text/event-stream', 'X-Accel-Buffering' => 'no']);
}

The X-Accel-Buffering: no header is essential when Nginx sits in front — without it, Nginx buffers the whole response.

Token Budget Guard

Never let user input dictate prompt size. Enforce a budget before the HTTP call.

// app/AI/TokenBudget.php
final class TokenBudget
{
    private const CHARS_PER_TOKEN = 4; // rough heuristic
    private const MAX_INPUT_TOKENS = 1_500;

    public static function enforce(string $text): string
    {
        $limit = self::MAX_INPUT_TOKENS * self::CHARS_PER_TOKEN;

        if (strlen($text) <= $limit) {
            return $text;
        }

        // Hard truncate, then append ellipsis so the model knows it's partial
        return mb_substr($text, 0, $limit) . ' [truncated]';
    }
}

For tool-calling agents, also set max_tokens on the API call itself — this is your last line of defence against runaway completions.

Structured Output Contracts

OpenAI's response_format: json_schema (available on gpt-4o and gpt-4o-mini) lets you pin the model to a JSON schema. Pair it with a PHP DTO and a single validation step.

// app/AI/Contracts/SentimentResult.php
final readonly class SentimentResult
{
    public function __construct(
        public string $label,   // positive | neutral | negative
        public float  $score,   // 0.0 – 1.0
        public string $reason,
    ) {}

    public static function fromArray(array $data): self
    {
        return new self(
            label:  $data['label'],
            score:  (float) $data['score'],
            reason: $data['reason'],
        );
    }
}

// app/AI/Agents/SentimentAgent.php
final class SentimentAgent
{
    public function analyse(string $text): SentimentResult
    {
        $text = TokenBudget::enforce($text);

        $response = Http::withToken(config('services.openai.key'))
            ->post('https://api.openai.com/v1/chat/completions', [
                'model'           => 'gpt-4o-mini',
                'max_tokens'      => 256,
                'response_format' => [
                    'type'        => 'json_schema',
                    'json_schema' => [
                        'name'   => 'sentiment_result',
                        'strict' => true,
                        'schema' => [
                            'type'       => 'object',
                            'properties' => [
                                'label'  => ['type' => 'string', 'enum' => ['positive','neutral','negative']],
                                'score'  => ['type' => 'number'],
                                'reason' => ['type' => 'string'],
                            ],
                            'required'            => ['label','score','reason'],
                            'additionalProperties'=> false,
                        ],
                    ],
                ],
                'messages' => [
                    ['role' => 'system', 'content' => 'Analyse sentiment. Reply only with the JSON schema.'],
                    ['role' => 'user',   'content' => $text],
                ],
            ])->throw()->json();

        $raw = json_decode(
            $response['choices'][0]['message']['content'],
            true,
            flags: JSON_THROW_ON_ERROR
        );

        return SentimentResult::fromArray($raw);
    }
}

With strict: true the model is constrained to the schema at the API level — you still validate on your side, but you'll rarely see a mismatch.

Keeping the Domain Clean

The SentimentAgent returns a typed DTO. Nothing in your domain layer touches raw LLM strings. If you swap providers tomorrow, only the agent changes — every caller keeps working.

Wrap the agent in a queued job for non-interactive workloads, and inject it via the service container so Pest can swap in a fake:

// tests/Feature/SentimentTest.php
it('classifies positive text', function () {
    $this->instance(SentimentAgent::class, new class {
        public function analyse(string $text): SentimentResult {
            return new SentimentResult('positive', 0.95, 'stub');
        }
    });

    $result = app(SentimentAgent::class)->analyse('Great product!');
    expect($result->label)->toBe('positive');
});

Takeaways

• Stream via response()->stream() and set X-Accel-Buffering: no for Nginx.
• Enforce token budgets before the HTTP call, not after.
• Use response_format.json_schema with strict: true to pin model output shape.
• Map LLM JSON to typed readonly DTOs immediately — keep raw strings out of your domain.
• Inject agents through the container so tests can swap fakes without HTTP calls.