Laravel Queues: Reliable Retry Strategies, Dead-Letter Handling, and Observability
#laravel #queues #reliability #observability #php

Laravel Queues: Reliable Retry Strategies, Dead-Letter Handling, and Observability

4 min read Mohamed Said Mohamed Said

The Problem With Default Queue Configuration

Laravel ships with sensible queue defaults, but "sensible" rarely means "production-ready". The default tries = 1, no backoff, and a single failed_jobs table give you a place to store failures — not a strategy for recovering from them. This article focuses on three concrete improvements: structured retry policies, a proper dead-letter pattern, and lightweight observability without a full APM stack.


Retry Policies That Actually Match Failure Modes

Not all failures are equal. A transient HTTP 429 from a third-party API needs exponential backoff. A validation failure should never retry at all. Encode that distinction directly in the job.

final class SyncExternalOrderJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 5;
    public int $maxExceptions = 3;

    public function backoff(): array
    {
        // Exponential: 10s, 60s, 300s, 600s
        return [10, 60, 300, 600];
    }

    public function handle(OrderSyncService $service): void
    {
        try {
            $service->sync($this->orderId);
        } catch (ValidationException $e) {
            // Permanent failure — do not retry
            $this->fail($e);
        }
    }
}

$maxExceptions caps retries on uncaught exceptions independently of $tries, which is useful when you want to allow manual releases ($this->release()) without burning retry budget on expected waits.


Implementing a Dead-Letter Queue

Laravel's failed_jobs table is a graveyard, not a queue. A dead-letter queue (DLQ) is a real queue you can inspect, replay, and alert on.

// In your job's failed() method
public function failed(Throwable $e): void
{
    DeadLetterJob::dispatch([
        'original_job' => static::class,
        'payload' => $this->toArray(),
        'exception' => $e->getMessage(),
        'failed_at' => now()->toIso8601String(),
    ])->onQueue('dead-letter');

    Log::channel('slack')->critical('Job permanently failed', [
        'job' => static::class,
        'order_id' => $this->orderId,
        'error' => $e->getMessage(),
    ]);
}

DeadLetterJob is a simple passthrough that stores the serialized payload in a dedicated table or Redis stream. You can replay it with a custom Artisan command:

protected function handle(): void
{
    DeadLetterEntry::unresolved()->each(function (DeadLetterEntry $entry) {
        $jobClass = $entry->original_job;
        $jobClass::dispatch(...$entry->reconstructedArgs())
            ->onQueue('default');
        $entry->markReplayed();
    });
}

This keeps your failed_jobs table for diagnostics and your DLQ for operational recovery — two different concerns.


Structured Observability Without a Full APM

You don't need Datadog to know what your queues are doing. Laravel's queue events give you everything you need to emit structured logs.

// AppServiceProvider::boot()
Queue::before(function (JobProcessing $event) {
    Log::info('job.started', [
        'job' => $event->job->resolveName(),
        'queue' => $event->job->getQueue(),
        'attempt' => $event->job->attempts(),
    ]);
});

Queue::after(function (JobProcessed $event) {
    Log::info('job.completed', [
        'job' => $event->job->resolveName(),
        'duration_ms' => /* track via context */ null,
    ]);
});

Queue::failing(function (JobFailed $event) {
    Log::error('job.failed', [
        'job' => $event->job->resolveName(),
        'exception' => $event->exception->getMessage(),
        'attempt' => $event->job->attempts(),
    ]);
});

Pair this with a log aggregator (Loki, Papertrail, CloudWatch) and you get queue throughput, failure rates, and attempt distributions without any additional dependencies.

Tracking Duration Properly

Use a request-scoped context value to measure wall time:

Queue::before(fn () => app()->instance('job.start', microtime(true)));

Queue::after(function (JobProcessed $event) {
    $duration = (microtime(true) - app('job.start')) * 1000;
    Log::info('job.completed', [
        'job' => $event->job->resolveName(),
        'duration_ms' => round($duration, 2),
    ]);
});

Because Octane workers are long-lived, always rebind job.start in before — never rely on a static property.


Key Takeaways

  • Match retry policy to failure mode: use backoff() arrays for exponential delays, $maxExceptions for exception caps, and $this->fail() for permanent failures.
  • Separate diagnostics from recovery: failed_jobs is for humans; a DLQ is for automated replay pipelines.
  • Queue events are free observability: Queue::before/after/failing emit structured logs with zero overhead.
  • Avoid static state in workers: always rebind per-job context in Queue::before when running under Octane or Swoole.
  • Test failure paths explicitly: assert that failed() dispatches to your DLQ in Pest using Queue::fake().

Found this useful?

Frequently Asked Questions

3 questions
Q01 What is the difference between `$tries` and `$maxExceptions` in a Laravel job?
`$tries` is the total number of times a job may be attempted, including manual `release()` calls. `$maxExceptions` limits how many times an *unhandled exception* can occur before the job is permanently failed, independent of the attempt count. Use both together when you want to allow deliberate delays via `release()` without burning your exception budget.
Q02 Should I use the `failed_jobs` table or a dedicated dead-letter queue?
Use both for different purposes. `failed_jobs` is a diagnostic record — it tells you what failed and why. A dead-letter queue is an operational tool: it holds serializable payloads you can replay, filter, and alert on programmatically. The `failed_jobs` table has no built-in replay mechanism beyond `queue:retry`, which doesn't scale for complex recovery workflows.
Q03 Are Queue event listeners safe to use under Laravel Octane?
Yes, but be careful with any state you bind into the container inside those listeners. Because Octane workers are long-lived, anything bound in `Queue::before` must be rebound on every job — never assume the previous job's context has been cleared. Binding a fresh value in `before` and reading it in `after` is safe as long as you don't use static properties or module-level variables.

Continue reading

More Articles

View all