Laravel Queues: Reliable Job Retry Strategies with Exponential Backoff and Dead-Letter Handling
#laravel #queues #reliability #backend

Laravel Queues: Reliable Job Retry Strategies with Exponential Backoff and Dead-Letter Handling

4 min read Mohamed Said Mohamed Said

Why Default Retry Behaviour Isn't Enough

Laravel's queue system ships with $tries and $backoff on every job. Most teams set $tries = 3 and call it done. That works until you hit a flaky third-party API, a brief database overload, or a downstream service that needs 30 seconds to recover — not 3 seconds.

This article covers three concrete improvements: exponential backoff with jitter, per-exception retry logic, and a dead-letter pattern that keeps failed jobs observable and replayable.


Exponential Backoff with Jitter

A flat $backoff = 5 means every retry hammers the same resource at the same cadence. Exponential backoff spreads load; jitter prevents the thundering-herd problem when many jobs fail simultaneously.

class SyncOrderToWarehouse implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 6;
    public int $maxExceptions = 3;

    // Called by Laravel to determine delay before each attempt.
    public function backoff(): array
    {
        return [
            $this->jitter(10),   // attempt 2: ~10s
            $this->jitter(30),   // attempt 3: ~30s
            $this->jitter(60),   // attempt 4: ~60s
            $this->jitter(120),  // attempt 5: ~120s
            $this->jitter(300),  // attempt 6: ~300s
        ];
    }

    private function jitter(int $base): int
    {
        return $base + random_int(0, (int) ($base * 0.2));
    }

    public function handle(WarehouseClient $client): void
    {
        $client->sync($this->order);
    }
}

Returning an array from backoff() maps each value to the corresponding retry attempt. Laravel falls back to the last value for any remaining attempts beyond the array length.


Per-Exception Retry Logic

Not all exceptions are equal. A RateLimitException deserves a long wait; a ValidationException should fail immediately without retrying at all.

public function handle(WarehouseClient $client): void
{
    try {
        $client->sync($this->order);
    } catch (RateLimitException $e) {
        // Re-release with a specific delay, not the backoff schedule.
        $this->release($e->retryAfter());
    } catch (\InvalidArgumentException $e) {
        // Permanent failure — don't retry, go straight to failed table.
        $this->fail($e);
    }
}

$this->release(int $delay) puts the job back on the queue with a custom delay without consuming a retry attempt. $this->fail(Throwable $e) marks the job failed immediately, bypassing remaining tries.


Dead-Letter Queue Pattern

Laravel's failed_jobs table is a dead-letter store, but it's passive. A production system needs active monitoring and a replay path.

Step 1 — Custom Failed Job Handler

Register a callback in AppServiceProvider:

Queue::failing(function (JobFailed $event) {
    Log::critical('Job permanently failed', [
        'job'        => $event->job->getName(),
        'connection' => $event->connectionName,
        'queue'      => $event->job->getQueue(),
        'payload'    => $event->job->payload(),
        'exception'  => $event->exception->getMessage(),
    ]);

    // Optionally push to a dedicated dead-letter queue for inspection.
    dispatch(new DeadLetterJob($event->job->payload()))
        ->onQueue('dead-letter');
});

Step 2 — Replay Command

class ReplayDeadLetterCommand extends Command
{
    protected $signature = 'queue:replay-dead-letter {--limit=50}';

    public function handle(): void
    {
        DB::table('failed_jobs')
            ->latest()
            ->limit((int) $this->option('limit'))
            ->get()
            ->each(function (object $row) {
                Artisan::call('queue:retry', ['id' => [$row->uuid]]);
                $this->line("Retried: {$row->uuid}");
            });
    }
}

Pair this with a Filament resource over failed_jobs for a UI-driven replay workflow.


$maxExceptions vs $tries

These two properties are frequently confused:

| Property | Meaning | |---|---| | $tries | Maximum total attempts (including first run) | | $maxExceptions | Max unhandled exceptions before marking failed, regardless of $tries |

Set $maxExceptions lower than $tries when you use $this->release() manually — otherwise a job that keeps rate-limiting itself will never count those releases against $tries, but unhandled exceptions will still accumulate.


Key Takeaways

  • Return an array from backoff() to define per-attempt delays; add jitter to avoid thundering herds.
  • Use $this->release($delay) for recoverable waits and $this->fail($e) for permanent errors — both bypass the default backoff schedule.
  • $maxExceptions caps unhandled exceptions independently of $tries; understand the distinction before combining them.
  • A Queue::failing() callback turns the passive failed_jobs table into an active alerting and dead-letter pipeline.
  • A replay command over failed_jobs gives you a safe, auditable path back to production queues.

Found this useful?

Frequently Asked Questions

3 questions
Q01 What is the difference between $tries and $maxExceptions on a Laravel job?
$tries is the total number of attempts Laravel will make, including the first run. $maxExceptions counts only unhandled exceptions; calls to $this->release() do not increment it. A job can be released many times without consuming $maxExceptions, but each unhandled throw does.
Q02 Does adding jitter to backoff() actually help in production?
Yes. When a downstream service fails, many queued jobs often fail at the same moment. Without jitter, all retries fire at identical intervals, recreating the same spike. Adding a small random offset (10–20% of the base delay) spreads retries across time and reduces the chance of overloading the recovering service again.
Q03 How do I prevent a job from retrying on a specific exception type?
Catch the exception inside handle() and call $this->fail($exception). This marks the job as permanently failed immediately, skipping all remaining retry attempts and backoff delays, and records it in the failed_jobs table.

Continue reading

More Articles

View all