For a while my way of chaining steps in AWS was the obvious one: a Lambda does its part and on finishing invokes the next, which invokes the next. It works with two steps. With six, with retries, with a step that sometimes takes ten minutes, and with the need to know where a process that failed last night got stuck, that approach becomes a tangle impossible to debug. Step Functions changed that for me: I pulled the coordination logic out of the code and into an explicit state machine. I chose this model in several flows because the cost of not knowing where each execution was outweighed the alternatives; in exchange I accepted learning a new definition language and paying per state transition. This article is about when it pays off and when it doesn't.
The problem: orchestration hidden in the code
When one Lambda invokes the next, the process logic —the order, the retries, what to do if step three fails— lives scattered across functions. There's no single place to read "this is how the whole flow works." To understand it you open five files and reconstruct the graph in your head.
The day that process fails in production halfway through, the key question is "which step did it get stuck on and with what data?". With chained Lambdas, the answer is buried in logs from five different functions that you have to correlate by hand. With an orchestrator, the answer is a screen: you see the execution, the exact state where it died, and the input it received.
The state machine as executable documentation
What I value most about Step Functions is that the flow definition is the diagram. There's no Confluence doc going stale: the source of truth for the process is the state machine running in production.
{
"StartAt": "ValidateOrder",
"States": {
"ValidateOrder": {
"Type": "Task",
"Resource": "arn:aws:lambda:...:validate",
"Retry": [{ "ErrorEquals": ["Timeout"], "MaxAttempts": 3 }],
"Next": "ChargePayment"
},
"ChargePayment": {
"Type": "Task",
"Resource": "arn:aws:lambda:...:charge",
"Catch": [{ "ErrorEquals": ["States.ALL"], "Next": "Compensate" }],
"Next": "Confirm"
},
"Compensate": { "Type": "Task", "Resource": "arn:aws:lambda:...:revert", "End": true },
"Confirm": { "Type": "Task", "Resource": "arn:aws:lambda:...:confirm", "End": true }
}
}
Notice where the retry and compensation policies live: in the definition, declarative, not scattered across try/catch in every function. Each Lambda goes back to doing a single thing and the orchestrator handles the rest. That's the split of responsibilities I was after.
Retries and compensation without hand-writing them
The reason I migrate flows to Step Functions most often is failure handling. In chained Lambdas, retrying with backoff, distinguishing transient from permanent errors, and compensating already-executed steps when something fails halfway is code you write, test, and maintain yourself, badly, in every function.
In the state machine, Retry and Catch are part of the definition. A timeout retries three times with exponential backoff without me writing a loop. And the saga pattern —if payment fails after reserving inventory, revert the reservation— is expressed with a compensation state you jump to with Catch, instead of with flags scattered across half the backend.
💡 The question I use to decide: do I need to know the exact step where a failed execution got stuck, weeks later? If the answer is yes, explicit orchestration pays for itself in the first incident.
Where I do NOT use Step Functions
Being honest about the trade-offs is what separates a decision from an act of faith. I don't put Step Functions in everything.
For a simple synchronous flow where the user waits for a response on screen, an orchestrator only adds latency and a layer nobody asked for; a direct Lambda is right. I also don't use it for very high-volume, short-duration processes where every state transition is billed: the Standard model charges per transition and at millions of short executions that scales into your bill in surprising ways. For that case I look at Express, which changes the pricing model and the durability guarantee, or straight to an SQS queue with a consumer that does all the work at once.
And there's a human cost: the definition language is one more thing the team has to learn and read. If your flow has two steps and will never grow, that curve isn't worth it.
The real price I paid
The most expensive part of adopting Step Functions wasn't the service, it was rewiring the team's mindset. We went from "the logic is in the code" to "the logic is in the state definition," and for a while people kept putting coordination inside the Lambdas out of habit, duplicating what the machine already did. The rule I set was clear: Lambdas do work, the state machine decides the order. Once the team internalized that boundary, debugging a long process stopped being log archaeology and became looking at a diagram that tells you exactly where you are. That shift, not the JSON syntax, is what makes it worth it.
