Job Batching, Chaining, and Rate-Limited Middleware in Laravel Queues

Why Basic dispatch() Is Not Enough

Single-job dispatching works fine for isolated tasks, but real SaaS workloads demand coordination: import a CSV, notify each row's owner, then send a summary email. Get any step wrong and you want partial retries — not a full restart. Laravel's batch and chain APIs, combined with rate-limited job middleware, give you that control.

Job Batching with Bus::batch()

Batches let you dispatch a collection of jobs and react when the whole set finishes, partially fails, or is cancelled.

use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;

$batch = Bus::batch([
    new ProcessRowJob($row) for $row in $rows, // spread or array
])
->then(fn (Batch $batch) => SummaryMail::dispatch($batch->id))
->catch(fn (Batch $batch, Throwable $e) => Log::error('Batch failed', [
    'batch' => $batch->id,
    'error' => $e->getMessage(),
]))
->finally(fn (Batch $batch) => BatchCompleted::dispatch($batch->id))
->name('csv-import')
->allowFailures()   // keep running even if some jobs fail
->dispatch();

allowFailures() is critical for large imports: one bad row should not cancel 10,000 others.

Track progress in Filament or a dashboard via $batch->progress(), $batch->failedJobs, and $batch->pendingJobs.

Adding Jobs to a Running Batch

Inside a batched job you can append more work — useful for tree-shaped workloads:

public function handle(): void
{
    $this->batch()->add([
        new ProcessChildJob($this->parentId, $child)
        foreach ($this->children() as $child),
    ]);
}

Job Chaining with Bus::chain()

Chains enforce strict sequential execution. If any job fails, the rest are abandoned.

Bus::chain([
    new VerifyPayment($orderId),
    new FulfillOrder($orderId),
    new SendConfirmationEmail($orderId),
])
->catch(fn (Throwable $e) => Order::fail($orderId, $e->getMessage()))
->dispatch();

You can mix batches inside chains for fan-out/fan-in patterns:

Bus::chain([
    new PrepareImport($fileId),
    Bus::batch($rowJobs)->allowFailures(),
    new FinaliseImport($fileId),
])->dispatch();

This runs PrepareImport, then all row jobs in parallel, then FinaliseImport — a powerful pattern for ETL pipelines.

Rate-Limited Job Middleware

Throttling at the job level prevents hammering third-party APIs regardless of how many workers you run.

use Illuminate\Queue\Middleware\RateLimited;
use Illuminate\Support\Facades\RateLimiter;

// AppServiceProvider::boot()
RateLimiter::for('stripe', fn () =>
    Limit::perMinute(100)->by('stripe-api')
);

// Inside the job
public function middleware(): array
{
    return [new RateLimited('stripe')];
}

When the limit is hit, the job is automatically released back to the queue with an exponential backoff — no manual $this->release() needed.

Custom Backoff on Rate Limit

use Illuminate\Queue\Middleware\RateLimitedWithRedis;

public function middleware(): array
{
    return [(new RateLimitedWithRedis('stripe'))->dontRelease()];
    // dontRelease() deletes the job instead of re-queuing — use carefully
}

RateLimitedWithRedis uses atomic Lua scripts for precise per-second limits, making it safer under Horizon's multi-worker concurrency.

Combining All Three

A production import pipeline might look like:

Bus::chain([
    new ValidateFile($fileId),                        // sequential
    Bus::batch($parseJobs)->allowFailures(),           // parallel parse
    Bus::batch($enrichJobs)->allowFailures(),          // parallel API calls (rate-limited)
    new GenerateReport($fileId),                      // sequential
])->dispatch();

Each enrichJob carries RateLimited('external-api') middleware, so the batch fans out as fast as the limiter allows without a single line of throttle logic in the business code.

Key Takeaways

• Use Bus::batch() for parallel fan-out; use allowFailures() for fault-tolerant imports.
• Use Bus::chain() for strict sequential steps; nest batches inside chains for fan-out/fan-in.
• Attach RateLimited middleware at the job level — it survives worker restarts and scales across all Horizon processes.
• Prefer RateLimitedWithRedis over the plain variant when you need sub-second precision.
• Track batch state via $batch->progress() for real-time dashboards without polling your database directly.