Laravel Horizon: Queue Metrics, Supervisor Tuning, and Reliable Job Throughput
#laravel #horizon #queues #performance

Laravel Horizon: Queue Metrics, Supervisor Tuning, and Reliable Job Throughput

3 min read Mohamed Said Mohamed Said

Laravel Horizon: Beyond the Default Config

Most teams install Horizon, paste the default config/horizon.php, and call it done. That works until traffic spikes, a slow third-party API blocks your default queue, and your critical payment jobs sit waiting behind a backlog of email sends.

This article covers three concrete improvements: priority-aware supervisor configuration, reading Horizon's metrics API to drive alerting, and writing jobs that survive unexpected worker termination.


Supervisor Configuration That Actually Reflects Priority

Horizon's supervisors key maps directly to separate worker pools. The key insight is that each supervisor is an independent process group — they do not share a queue slot budget.

// config/horizon.php
'environments' => [
    'production' => [
        'supervisor-critical' => [
            'connection' => 'redis',
            'queue' => ['critical'],
            'balance' => 'simple',
            'minProcesses' => 3,
            'maxProcesses' => 10,
            'tries' => 3,
            'timeout' => 30,
        ],
        'supervisor-default' => [
            'connection' => 'redis',
            'queue' => ['default', 'notifications'],
            'balance' => 'auto',
            'minProcesses' => 2,
            'maxProcesses' => 20,
            'balanceCooldown' => 3,
            'tries' => 5,
            'timeout' => 90,
        ],
    ],
],

balance => auto lets Horizon scale workers proportionally to queue depth. Use simple for critical queues where you want a fixed floor of workers always ready, not waiting for the autoscaler to react.

balanceCooldown (seconds) prevents thrashing — Horizon won't add or remove workers more often than this interval. Set it higher (5–10) on queues with bursty but short-lived spikes.


Reading Horizon Metrics Programmatically

Horizon stores throughput and runtime metrics in Redis. You can read them via the Metrics repository to drive custom dashboards or Slack alerts.

use Laravel\Horizon\Contracts\MetricsRepository;

$metrics = app(MetricsRepository::class);

$throughput = $metrics->throughputForQueue('critical'); // jobs/min
$runtime    = $metrics->runtimeForQueue('critical');    // ms average

if ($throughput < 10 && $runtime > 5000) {
    // Alert: critical queue is slow and under-processed
    Log::channel('slack')->critical('Horizon critical queue degraded', [
        'throughput' => $throughput,
        'avg_runtime_ms' => $runtime,
    ]);
}

Wrap this in a scheduled command running every minute:

// routes/console.php
Schedule::command('horizon:metrics:check')->everyMinute()->withoutOverlapping();

This gives you alerting without a paid APM tool for queue-specific health.


Jobs That Survive Worker Restarts

When you run horizon:terminate or deploy a new release, Horizon sends SIGTERM to workers. A job mid-execution gets re-queued if it hasn't been deleted — but only if you haven't already committed a side effect.

The Pattern: Idempotency Key + Atomic Check

class ChargeSubscription implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public string $idempotencyKey;

    public function __construct(public int $subscriptionId)
    {
        $this->idempotencyKey = 'charge:' . $subscriptionId . ':' . now()->format('Y-m-d');
    }

    public function handle(PaymentGateway $gateway): void
    {
        $alreadyCharged = Cache::add(
            $this->idempotencyKey,
            true,
            now()->addDay()
        );

        if (! $alreadyCharged) {
            return; // Re-queued after SIGTERM — skip silently
        }

        $gateway->charge($this->subscriptionId);
    }
}

Cache::add is atomic on Redis — it only sets the key if it does not exist, returning false if the key was already present. This prevents double-charges when a job is re-queued after an interrupted worker.


Takeaways

  • Separate supervisors per priority — never let low-priority jobs compete for the same worker pool as critical ones.
  • Use balance => simple with a minProcesses floor for queues where latency matters more than cost.
  • MetricsRepository is public API — use it for custom alerting before reaching for an APM.
  • Cache::add is your idempotency primitive — combine it with a deterministic key to make re-queued jobs safe.
  • balanceCooldown prevents autoscaler thrashing — tune it per queue's burst profile, not globally.

Found this useful?

Frequently Asked Questions

3 questions
Q01 How does Horizon's `balance => auto` differ from `balance => simple`?
`auto` scales worker count proportionally to queue depth across all queues in that supervisor, redistributing processes dynamically. `simple` keeps a fixed number of workers per queue without redistribution. Use `simple` when you need a guaranteed minimum for latency-sensitive queues.
Q02 Will a job be lost if a Horizon worker receives SIGTERM mid-execution?
No — Horizon re-queues the job if it has not been deleted from Redis yet. The risk is duplicate execution, not data loss. Guard against duplicates with an idempotency key stored atomically in Redis via `Cache::add`.
Q03 Can I use `MetricsRepository` outside of Horizon's dashboard?
Yes. It is bound in the service container and can be resolved anywhere in your application. Horizon must be running and processing jobs for the metrics to be populated, but reading them is safe from any context including scheduled commands.

Continue reading

More Articles

View all