sunny34.com

1. Why one request should not try to finish the whole job

Agent tasks that read documents, call tools, and update external systems are often longer and less predictable than teams expect. If everything is forced into one synchronous request, timeouts and duplicate execution become normal failure modes.

A safer pattern is to separate intake, background processing, and final approval. Intake locks the request identity and inputs. Background processing accumulates agent work and tool results. Finalization applies any external write only after the right conditions are met.

2. Queue boundaries should follow recovery boundaries

Many teams split queues by feature labels. In practice, it is safer to split them by where recovery can restart. One queue might validate policy and normalize input. Another might handle model reasoning and tool calls. A final queue might perform the external write after approval.

• Intake queue: store the request, user identity, permissions, and idempotency key.
• Execution queue: track model output, tool calls, retries, and intermediate summaries.
• Finalization queue: write to CRM, CMS, or ticketing systems only after approval.

That structure protects service stability even when the model is imperfect.

3. Approval should be layered by risk

Human approval is not a binary switch. Some tasks can be fully automated through draft generation. Others should stop before any customer-facing message, payment change, or public publication. Approval layers should follow impact, not convenience.

The approval interface should also be designed for fast judgment. Show the summary, evidence, expected impact, and rollback path rather than exposing the entire prompt history.

4. Retries should distinguish model failure from system failure

Not every failure deserves an automatic retry. Network issues and temporary API delays often do. Policy violations, missing evidence, or conflicting instructions usually require review instead. Stable systems classify failure type before retrying.

At minimum, logs should capture job ID, prior state, current state, model version, tools used, retry count, and the final decision path. Without those fields, teams cannot explain where a job stopped or where it should resume.

5. Start with one high-risk workflow

The point of an asynchronous job queue is not to move every AI feature into a giant orchestration engine on day one. It is usually better to pick one high-risk, long-running flow first, prove that timeouts fall and approval quality improves, and then expand gradually.

In other words, the real gain is not making the model smarter. It is making failures, approvals, and restarts visible as state transitions humans can understand.