A queue isn't a luxury: it's how you pull everything that doesn't need an instant answer off the critical path. In JXBS I send to SQS whatever can wait a few seconds: candidate emails, match recomputation, syncs. What I learned the hard way is that the tricky part isn't enqueuing, it's making sure the retry doesn't redo the work. SQS delivers "at least once", and that "at least" is exactly where things break.
Why I enqueue instead of answering inline
When a recruiter posts a role, the HTTP response shouldn't wait for embeddings to be generated, matches recomputed, and notifications fired. I send that to a queue and return immediately. The user perceives a fast app; the heavy lifting happens behind the scenes, at its own pace.
The queue gives me three concrete things. It absorbs spikes: if a hundred roles come in at once, they're processed at the rate the consumer can handle, not all at once. It isolates failures: if the email provider is down, the task retries itself without taking down the original request. And it decouples: the producer neither knows nor cares who consumes.
The real problem: "at least once"
Standard SQS guarantees delivery, not uniqueness. The same message can arrive twice: because the consumer took longer than the visibility timeout and SQS redelivered, because of a network retry, or because the process died right after working but before deleting the message.
If my consumer just sends an email per message, a candidate gets two. If it creates a charge, I bill twice. Duplicate delivery isn't a rare edge case that happens once a year: it's the system's normal behavior, and you have to design assuming it will happen.
Idempotency: the only defense that scales
The fix isn't to avoid duplicates, it's to make processing twice yield the same result as processing once. That's idempotency, and I implement it with a stable key per unit of work.
CREATE TABLE processed_message (
idempotency_key TEXT PRIMARY KEY,
processed_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
Before doing any work, the consumer tries to insert the key. If it already exists, the message is a duplicate and I discard it with no side effects.
async function handle(msg: Job) {
const inserted = await db.processedMessage
.create({ data: { idempotencyKey: msg.id } })
.catch(() => null); // hits the PK if already processed
if (!inserted) return; // duplicate: do nothing
await doTheActualWork(msg);
}
I don't invent the key at random: I derive it from the message's intent. For a welcome email it's welcome-email:{candidateId}, not a fresh UUID per attempt. That way two messages representing the same action collide and only one wins.
💡 A safe retry isn't one that never repeats; it's one that can repeat without anyone caring.
Visibility timeout and the DLQ
Two SQS pieces need tuning. The visibility timeout must be longer than the realistic worst-case processing time: if I take longer, SQS assumes I failed and redelivers while I'm still working. I size it against the consumer's p99, not the average.
And the dead-letter queue: after N failed attempts, the message leaves the main queue and lands in a DLQ. Without it, a poison message —one that always fails— retries forever and clogs the queue. The DLQ is where I inspect what broke without it blocking everything else. I put an alarm on it: if something lands there, I want to know.
What I'd do differently
At first I put the idempotency logic inside each handler. I ended up repeating it and getting it wrong differently in each one. Today it lives in a single wrapper: it takes the key, checks, executes, and no handler thinks about duplicates again. The pattern matters more than the infrastructure. SQS is replaceable; the discipline of assuming every message can arrive twice is what actually saves you.
