Streaming AI in Laravel: Token Budgets & Structured Output | Mohamed Said [ ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MH.png) Mohamed Said Laravel Backend Engineer ](https://msaied.com) [ Home ](https://msaied.com) [ Projects ](https://msaied.com/projects) [ Articles ](https://msaied.com/articles) [ Certificates ](https://msaied.com/certificates) [ Contact ](https://msaied.com#contact-section) [ ](https://github.com/EG-Mohamed) [ Home ](https://msaied.com) [ Projects ](https://msaied.com/projects) [ Articles ](https://msaied.com/articles) [ Certificates ](https://msaied.com/certificates) [ Contact ](https://msaied.com#contact-section) [ home ](https://msaied.com) [ articles ](https://msaied.com/articles) Streaming AI Responses in Laravel: Token Budgets, Structured Output, and Production Contracts On this page 1. [ The Problem With Naive AI Integration ](#the-problem-with-naive-ai-integration) 2. [ 1. Streaming Completions Without Blocking the Worker ](#1-streaming-completions-without-blocking-the-worker) 3. [ 2. Enforcing Token Budgets ](#2-enforcing-token-budgets) 4. [ 3. Structured Output Contracts with JSON Schema ](#3-structured-output-contracts-with-json-schema) 5. [ 4. Octane Safety: No Static State, No Singleton Leakage ](#4-octane-safety-no-static-state-no-singleton-leakage) 6. [ Takeaways ](#takeaways) ![Streaming AI Responses in Laravel: Token Budgets, Structured Output, and Production Contracts](https://cdn.msaied.com/199/0042e3985f7cb3bfdaeb2d1c76c79489.png) #laravel #ai #streaming #php Streaming AI Responses in Laravel: Token Budgets, Structured Output, and Production Contracts =============================================================================================== 15 Jun 2026 4 min read ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said Table of contents 1. [ 01 The Problem With Naive AI Integration ](#the-problem-with-naive-ai-integration) 2. [ 02 1. Streaming Completions Without Blocking the Worker ](#1-streaming-completions-without-blocking-the-worker) 3. [ 03 2. Enforcing Token Budgets ](#2-enforcing-token-budgets) 4. [ 04 3. Structured Output Contracts with JSON Schema ](#3-structured-output-contracts-with-json-schema) 5. [ 05 4. Octane Safety: No Static State, No Singleton Leakage ](#4-octane-safety-no-static-state-no-singleton-leakage) 6. [ 06 Takeaways ](#takeaways) The Problem With Naive AI Integration ------------------------------------- Most Laravel AI tutorials end at `Http::post()` and `json_decode()`. In production you face three harder problems: responses that arrive token-by-token (streaming), models that hallucinate structure (unstructured output), and long-lived Octane workers that silently carry state between requests. This article tackles all three with concrete, opinionated patterns. --- 1. Streaming Completions Without Blocking the Worker ---------------------------------------------------- OpenAI's streaming API sends `text/event-stream` chunks. Laravel's HTTP client wraps Guzzle, so you can consume the stream lazily: ```php use Illuminate\Support\Facades\Http; function streamCompletion(string $prompt): \Generator { $response = Http::withToken(config('services.openai.key')) ->withOptions(['stream' => true]) ->post('https://api.openai.com/v1/chat/completions', [ 'model' => 'gpt-4o', 'stream' => true, 'max_tokens' => 512, 'messages' => [['role' => 'user', 'content' => $prompt]], ]); $body = $response->toPsrResponse()->getBody(); while (! $body->eof()) { $line = trim($body->read(4096)); if (str_starts_with($line, 'data: ') && $line !== 'data: [DONE]') { $chunk = json_decode(substr($line, 6), true); yield $chunk['choices'][0]['delta']['content'] ?? ''; } } } ``` Return this generator from a `StreamedResponse` so Nginx flushes each chunk immediately: ```php Route::get('/stream', function () { return response()->stream(function () { foreach (streamCompletion('Explain CQRS in one paragraph') as $token) { echo "data: {$token}\n\n"; ob_flush(); flush(); } }, 200, ['Content-Type' => 'text/event-stream', 'X-Accel-Buffering' => 'no']); }); ``` `X-Accel-Buffering: no` is mandatory when Nginx sits in front — without it the proxy buffers the entire response. --- 2. Enforcing Token Budgets -------------------------- `max_tokens` is a ceiling, not a guarantee. A budget-aware wrapper counts tokens before the call and aborts early if the prompt itself is too large: ```php final class TokenBudget { public function __construct( private readonly int $maxPromptTokens = 3_000, private readonly int $maxCompletionTokens = 512, ) {} /** Rough estimate: 1 token ≈ 4 chars for English prose */ public function promptFits(string $prompt): bool { return (int) ceil(mb_strlen($prompt) / 4) maxPromptTokens; } public function completionLimit(): int { return $this->maxCompletionTokens; } } ``` Bind it as a singleton in `AppServiceProvider` and inject it wherever you build prompts. This prevents runaway costs when user-supplied context is large. --- 3. Structured Output Contracts with JSON Schema ----------------------------------------------- OpenAI's `response_format` with `json_schema` mode guarantees the model returns valid JSON matching your schema — or it refuses rather than hallucinating: ```php $schema = [ 'type' => 'object', 'properties' => [ 'summary' => ['type' => 'string'], 'confidence' => ['type' => 'number', 'minimum' => 0, 'maximum' => 1], 'tags' => ['type' => 'array', 'items' => ['type' => 'string']], ], 'required' => ['summary', 'confidence', 'tags'], 'additionalProperties' => false, ]; $result = Http::withToken(config('services.openai.key')) ->post('https://api.openai.com/v1/chat/completions', [ 'model' => 'gpt-4o-2024-08-06', 'max_tokens' => 256, 'response_format' => [ 'type' => 'json_schema', 'json_schema' => ['name' => 'analysis', 'strict' => true, 'schema' => $schema], ], 'messages' => [['role' => 'user', 'content' => "Analyse: {$text}"]], ])->json('choices.0.message.content'); $dto = AnalysisResult::fromArray(json_decode($result, true)); ``` Map the validated JSON straight into a typed DTO — no defensive `isset()` chains needed. --- 4. Octane Safety: No Static State, No Singleton Leakage ------------------------------------------------------- Octane workers are long-lived. Any static property or singleton that accumulates per-request data will bleed across users. For AI work: - **Never** store conversation history in a singleton. Use the session or a database-backed `Conversation` model. - Bind AI client wrappers as `scoped()` (reset per request) rather than `singleton()`. - Use `defer()` for logging token usage so it runs after the response is sent. ```php // AppServiceProvider $this->app->scoped(AiClient::class, fn () => new AiClient( apiKey: config('services.openai.key'), )); ``` --- Takeaways --------- - Stream via `Http::withOptions(['stream' => true])` and yield chunks through a `StreamedResponse`. - Set `X-Accel-Buffering: no` when Nginx proxies the stream. - Estimate prompt token size before the API call to enforce hard cost budgets. - Use OpenAI's `json_schema` response format to get guaranteed-valid structured output. - Register AI clients as `scoped()` bindings in Octane to prevent cross-request state leakage. - Map structured responses directly into typed DTOs — skip defensive null-checking. Found this useful? [ ](https://twitter.com/intent/tweet?url=https%3A%2F%2Fmsaied.com%2Farticles%2Fstreaming-ai-responses-in-laravel-token-budgets-structured-output-and-production-contracts&text=Streaming+AI+Responses+in+Laravel%3A+Token+Budgets%2C+Structured+Output%2C+and+Production+Contracts) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fmsaied.com%2Farticles%2Fstreaming-ai-responses-in-laravel-token-budgets-structured-output-and-production-contracts) Frequently Asked Questions ---------------------------- 3 questions Q01 Does Laravel's HTTP client support streaming responses natively? Yes. Pass `['stream' => true]` in `withOptions()` and call `toPsrResponse()->getBody()` to get a PSR-7 stream you can read incrementally. Wrap the output in `response()->stream()` to flush tokens to the browser as they arrive. Q02 What is the difference between `max\_tokens` and a token budget? `max_tokens` caps the model's completion length on the API side. A token budget is an application-level guard that estimates prompt size before the call and rejects requests that would exceed your cost or context-window limits — preventing expensive or failed API calls entirely. Q03 How do I prevent AI client state from leaking between Octane requests? Register your AI client wrapper with `$this->app->scoped()` instead of `singleton()`. Scoped bindings are flushed and rebuilt at the start of each request cycle, so no per-request data (conversation history, accumulated tokens) survives into the next request. Continue reading More Articles --------------- [ View all ](https://msaied.com/articles) [ ![Laravel Eloquent Global Scopes: Pitfalls, Testing, and Composing Them Safely](https://cdn.msaied.com/211/8b9b19e7ecbf690b182ffbe6bffc9530.png) laravel eloquent testing ### Laravel Eloquent Global Scopes: Pitfalls, Testing, and Composing Them Safely Global scopes are powerful but easy to misuse. Learn how to write, test, and safely compose Eloquent global sc... ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said 16 Jun 2026 1 min read Read ](https://msaied.com/articles/laravel-eloquent-global-scopes-pitfalls-testing-and-composing-them-safely) [ ![Eloquent Custom Relations: Polymorphic Pivots, HasManyThrough Tricks, and Raw Join Relations](https://cdn.msaied.com/210/b47272214946c6adcd02ddf74b7df816.png) laravel eloquent database ### Eloquent Custom Relations: Polymorphic Pivots, HasManyThrough Tricks, and Raw Join Relations Beyond belongsTo and hasMany lies a set of underused Eloquent relation techniques. This guide covers custom re... ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said 16 Jun 2026 3 min read Read ](https://msaied.com/articles/eloquent-custom-relations-polymorphic-pivots-hasmanythrough-tricks-and-raw-join-relations) [ ![New in Laravel 12: Features, Helpers, and Upgrade Notes](https://cdn.msaied.com/209/c713447686bc1eb0a921b4027e4e4df8.png) laravel php upgrade ### New in Laravel 12: Features, Helpers, and Upgrade Notes Laravel 12 ships with a refined starter kit system, per-request context propagation, and several quality-of-li... ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said 16 Jun 2026 3 min read Read ](https://msaied.com/articles/new-in-laravel-12-features-helpers-and-upgrade-notes) [ ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MH.png) Mohamed Said Laravel Backend Engineer ](https://msaied.com)Senior Backend Engineer specializing in Laravel, scalable SaaS platforms, APIs, and cloud infrastructure. I build secure, high-performance web applications that help businesses grow. Explore - [Home](https://msaied.com) - [Projects](https://msaied.com/projects) - [Articles](https://msaied.com/articles) - [Certificates](https://msaied.com/certificates) - [Contact](https://msaied.com#contact-section) Connect - [ hello@msaied.com ](mailto:hello@msaied.com) - [ +20 109 461 9204 ](tel:+201094619204) © 2026 Mohamed Said. All rights reserved. [ ](https://github.com/EG-Mohamed) [ ](https://www.linkedin.com/in/msaiedm/) [ ](https://wa.me/201094619204) [ ](mailto:hello@msaied.com) [ ](https://drive.google.com/file/u/0/d/1MF20IPRJyzfy32mhEutjL5EpSls0w2Q8/view)