Your Queue Retry Policy Is a Data Loss Bug

A job fails because an upstream API times out. The queue retries it. The API had actually processed the request and the response was lost in transit. Now the customer has been charged twice, and the retry policy that was supposed to make the system resilient is the thing that broke it.

Retries are not a resilience feature

They are a duplicate delivery feature. That is their entire mechanism. Every retry is a second chance for the side effect to fire, and the framework has no way to know whether it already did.

This is the same problem distributed systems have had forever — at-least-once delivery — and the only real solution is to make the consumer handle duplicates correctly.

Making a job idempotent

The principle is to derive an idempotency key from the intent of the operation, persist it before the side effect, and check it before repeating the work.

final class ChargeCustomer implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

public int $tries = 5;
public array $backoff = [10, 30, 120, 600];

public function __construct(
public readonly int $invoiceId,
public readonly string $idempotencyKey,
) {}

public function handle(PaymentGateway $gateway): void
{
$alreadyCharged = DB::transaction(function (): bool {
$row = ChargeAttempt::query()
->where('idempotency_key', $this->idempotencyKey)
->lockForUpdate()
->first();

if ($row !== null && $row->completed_at !== null) {
return true;
}

ChargeAttempt::query()->updateOrCreate(
['idempotencykey' => $this->idempotencyKey],
['invoiceid' => $this->invoiceId, 'attempted_at' => now()],
);

return false;
});

if ($alreadyCharged) {
return; // Safe no-op: a previous attempt succeeded.
}

$result = $gateway->charge(
invoiceId: $this->invoiceId,
idempotencyKey: $this->idempotencyKey,
);

ChargeAttempt::query()
->where('idempotencykey', $this->idempotencyKey)
->update(['completedat' => now(), 'reference' => $result->reference]);
}
}
```

Three properties make this work:

1. The key is derived from intent, not generated per attempt. "invoice:{$invoiceId}:charge" is stable across retries; a UUID generated in handle() is not.
2. The record is written inside a transaction with a row lock, so two concurrent workers cannot both pass the check.
3. The consumer is safe to run twice, so the retry policy becomes what it should have been all along — a way to recover from transient failures rather than a source of new ones.

The idempotency key is not the same as the job id

Laravel gives every job a unique id. It is tempting to use it as the key. Do not: retries of the same job share an id, but a re-dispatch after a failure — which is what happens when you call ->release() or a supervisor restarts the worker — does not. A key derived from the job id therefore gives you the duplicate protection you expect in one path and none in the other, which is worse than no protection because it looks like it works.

Where the key comes from

| Operation | Key shape | Why |
| --- | --- | --- |
| Charge an invoice | invoice:{id}:charge | One charge per invoice, ever |
| Send a receipt | invoice:{id}:receipt:{email} | One receipt per recipient, re-sendable to a new address |
| Sync a record | sync:{resource}:{id}:{version} | Re-syncing the same version is a no-op |
| Send a welcome email | user:{id}:welcome | Once per user |

The key encodes the thing that must happen once, not the attempt.

What to do about jobs you cannot make idempotent

Some side effects genuinely cannot be de-duplicated — sending an SMS, calling a third party with no idempotency support. For those:

• Set $tries = 1 and accept the failure. A manual retry with human judgement is better than an automatic one that duplicates.
• Or move the deduplication outward: check "was this sent" in your own records before calling, and accept that the window between check and call is a real risk you have documented rather than ignored.

What you must not do is leave $tries at the default and hope.

The audit I run on every new queue

Four questions, in order:

1. What happens if this job runs twice?
2. What happens if it runs twice concurrently?
3. What happens if it fails after the side effect but before the bookkeeping?
4. What is the idempotency key, and is it derived from intent rather than from the attempt?

If any answer is "I am not sure," the retry policy is currently a data-loss bug waiting for the right network blip.