Job Batching, Chaining, and Rate-Limited Middleware in Laravel Queues

Job Batching, Chaining, and Rate-Limited Middleware in Laravel Queues

Laravel's queue system is deceptively deep. Most teams dispatch individual jobs and call it done. But production workflows demand composition: run a set of jobs in parallel, then trigger a follow-up only when all succeed, and never hammer a third-party API faster than its rate limit allows. Laravel gives you all three primitives — batches, chains, and job middleware — and combining them correctly is where the real power lives.

Batches: Parallel Work With a Shared Lifecycle

A batch dispatches multiple jobs concurrently and exposes callbacks for completion, failure, and progress.

use Illuminate\Support\Facades\Bus;

$batch = Bus::batch([
    new ProcessInvoice($invoice1),
    new ProcessInvoice($invoice2),
    new ProcessInvoice($invoice3),
])
->then(fn (Batch $batch) => SendBatchSummary::dispatch($batch->id))
->catch(fn (Batch $batch, Throwable $e) => Log::error('Batch failed', [
    'batch' => $batch->id,
    'error' => $e->getMessage(),
]))
->finally(fn (Batch $batch) => Cache::forget('invoices:processing'))
->name('Invoice Processing')
->allowFailures() // don't cancel remaining jobs on a single failure
->dispatch();

Store $batch->id if you need to poll progress from a UI. $batch->progress() returns 0–100 based on processed job count.

Important: Every job in a batch must use the Batchable trait, and you should check $this->batch()->cancelled() at the start of handle() to respect early cancellation.

use Illuminate\Bus\Batchable;
use Illuminate\Contracts\Queue\ShouldQueue;

class ProcessInvoice implements ShouldQueue
{
    use Batchable, Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function handle(): void
    {
        if ($this->batch()?->cancelled()) {
            return;
        }

        // ... process
    }
}

Chains: Sequential Pipelines With Shared Context

Chains execute jobs one after another; if any job fails, the rest are abandoned.

Bus::chain([
    new ValidateOrder($order),
    new ChargePayment($order),
    new FulfillOrder($order),
    new SendConfirmationEmail($order),
])
->catch(fn (Throwable $e) => $order->markFailed($e->getMessage()))
->dispatch();

You can mix batches inside chains for hybrid workflows:

Bus::chain([
    new PrepareExport($report),
    Bus::batch([
        new ExportChunk($report, 0),
        new ExportChunk($report, 1),
        new ExportChunk($report, 2),
    ]),
    new MergeExportChunks($report),
])->dispatch();

This pattern — prepare, fan-out in parallel, then merge — is a clean map-reduce for queue workers.

Rate-Limited Job Middleware

Throttling at the job level is far more reliable than sleeping inside handle(). Laravel's RateLimited middleware uses the cache to enforce limits across all workers.

use Illuminate\Queue\Middleware\RateLimited;

class SyncToStripe implements ShouldQueue
{
    public function middleware(): array
    {
        return [new RateLimited('stripe-sync')];
    }
}

Define the limiter in a service provider:

use Illuminate\Cache\RateLimiting\Limit;
use Illuminate\Support\Facades\RateLimiter;

RateLimiter::for('stripe-sync', fn () =>
    Limit::perMinute(60)->response(fn () =>
        // job is released back to the queue automatically
        null
    )
);

When the limit is hit, the job is released back with a calculated backoff — no wasted worker time, no dropped requests.

For per-tenant limits, key the limiter dynamically:

public function middleware(): array
{
    return [new RateLimited('stripe-sync:' . $this->tenantId)];
}

Takeaways

• Use Bus::batch() for parallel fan-out; always add Batchable and check for cancellation.
• Use Bus::chain() for sequential pipelines where order and dependency matter.
• Nest a batch inside a chain for map-reduce style workflows.
• Throttle third-party API jobs with RateLimited middleware — never sleep().
• Key rate limiters per tenant or resource to avoid cross-tenant interference.
• Monitor batch progress via $batch->id and expose it to your UI for long-running operations.