Skip to content

Laravel Queues at Scale: Backpressure, Overflow Routing, and Dead-Letter Strategies

Beyond basic queue configuration lies a set of production patterns—backpressure detection, overflow routing to fallback queues, and structured dead-letter handling—that keep high-throughput Laravel apps stable under load.

Published · Updated · 3 min read

Laravel Queues at Scale: Backpressure, Overflow Routing, and Dead-Letter Strategies

The Problem Nobody Talks About

Most Laravel queue tutorials stop at php artisan queue:work and a basic Horizon config. That works fine until your queue depth spikes to 50 000 jobs and workers start thrashing, retrying aggressively, and hammering your database. The real craft is in backpressure, overflow routing, and dead-letter handling.


Detecting Backpressure Before It Hurts

Backpressure means your producers are outpacing your consumers. The earliest signal is queue depth. Poll it on a schedule and react before workers drown:

// app/Console/Commands/MonitorQueueDepth.php
public function handle(Queue $queue): void
{
    $depth = $queue->size('default');

    if ($depth > 10_000) {
        // Emit a metric, fire an alert, or flip a feature flag
        Cache::put('queue:backpressure', true, now()->addMinutes(5));
        Log::warning('Queue backpressure detected', ['depth' => $depth]);
    } else {
        Cache::forget('queue:backpressure');
    }
}

Schedule this every minute. Pair it with a Horizon metric or a Prometheus exporter so your alerting fires before users notice.


Overflow Routing: Shedding Load Gracefully

When backpressure is active, route lower-priority work to a slower overflow queue instead of blocking the primary one. A middleware on the job itself is the cleanest place to enforce this:

// app/Jobs/Middleware/OverflowRouter.php
class OverflowRouter
{
    public function handle(object $job, callable $next): void
    {
        if (Cache::get('queue:backpressure') && method_exists($job, 'onQueue')) {
            // Re-dispatch to overflow and bail out of current attempt
            dispatch($job)->onQueue('overflow');
            return;
        }

        $next($job);
    }
}

Attach it selectively to non-critical jobs:

public function middleware(): array
{
    return [new OverflowRouter];
}

In Horizon, give overflow a single slow worker so it drains without competing for resources:

// config/horizon.php
'overflow' => [
    'connection' => 'redis',
    'queue' => ['overflow'],
    'balance' => 'simple',
    'processes' => 1,
    'tries' => 3,
],

Structured Dead-Letter Handling

Laravel's failed_jobs table is a graveyard, not a strategy. Treat it as a first-class queue by wrapping failure logic in a dedicated handler.

Step 1 — Capture rich context on failure

// app/Jobs/Concerns/RecordsFailureContext.php
trait RecordsFailureContext
{
    public function failed(Throwable $e): void
    {
        DeadLetterEntry::create([
            'job_class' => static::class,
            'payload' => $this->toDeadLetterPayload(),
            'exception' => $e->getMessage(),
            'failed_at' => now(),
        ]);
    }

    abstract protected function toDeadLetterPayload(): array;
}

Step 2 — Replay from the dead-letter store

// app/Console/Commands/ReplayDeadLetters.php
public function handle(): void
{
    DeadLetterEntry::whereNull('replayed_at')
        ->chunkById(100, function ($entries) {
            foreach ($entries as $entry) {
                $job = new $entry->job_class(...$entry->payload);
                dispatch($job)->onQueue('overflow');
                $entry->update(['replayed_at' => now()]);
            }
        });
}

This gives you a controlled, auditable replay path instead of blindly running queue:retry all.


Combining the Pieces

The full flow looks like this:

  1. A scheduled command monitors queue depth every minute.
  2. When depth exceeds a threshold, a cache flag signals backpressure.
  3. Non-critical jobs detect the flag via middleware and re-dispatch to overflow.
  4. Jobs that exhaust retries write structured records to dead_letter_entries.
  5. An operator-triggered command replays dead letters through overflow at a controlled rate.

Takeaways

  • Poll queue depth on a schedule; don't wait for Horizon alerts alone.
  • Job middleware is the right abstraction for overflow routing — it keeps the logic out of your business code.
  • Dead-letter tables with rich context beat the default failed_jobs table for replay and auditing.
  • Overflow queues with a single worker act as a pressure valve without dropping work.
  • Chunk-based replay prevents a flood of retries from re-triggering the same backpressure you just escaped.

Frequently asked questions

Why not just increase worker count to handle backpressure?

Adding workers helps up to the point where your database or downstream services become the bottleneck. Beyond that, more workers increase contention and can make things worse. Overflow routing buys time without adding load to already-saturated resources.

Is there a risk of a job being dispatched twice with the OverflowRouter middleware?

Yes, if the middleware re-dispatches and then throws before returning, the original job may retry. Guard against this by catching dispatch exceptions and marking the job as released rather than failed, or by making your jobs idempotent using a unique ID stored in cache.

Can this pattern work with database queues instead of Redis?

Yes. Queue depth via `Queue::size()` works with the database driver. The overflow and dead-letter patterns are driver-agnostic. Redis is preferred at scale because depth queries are O(1) rather than a COUNT on a large table.

Mohamed Said

About the author

Senior Backend Engineer specializing in Laravel, scalable SaaS platforms, APIs, and cloud infrastructure. I build secure, high-performance web applications that help businesses grow.

About GitHub LinkedIn WhatsApp Email Address My CV

Have a technical challenge?

Tell me what you’re building. I reply within two working days.

Discuss your project