Agent Loop Economics: Why Run Mode Now Comes First
Decoding AI and Paul Iusztin’s open-source course, highlighted on 2026-08-22, mapped one agent loop into three run modes: interactive, remote offline, and async online. That matters because it shifts the main design question from model choice to runtime shape, latency tolerance, and provider cost. According to MarkTechPost’s coverage of the course, the economics change sharply depending on whether a human is waiting or a queue is doing the waiting.
Decoding AI’s three agent loop run modes
The news here is not that coding agents can call tools. Teams already know that. The new framing is that the same headless core can be presented in three operating shapes, each with a different performance target.
At the center is a compact loop: the model selects an action, a tool runs, and the observation returns to the context window. In the source material, Decode is described as a small Pydantic AI definition, while much of the real system complexity sits outside that core in memory, permissions, sandboxing, steering, and compaction.
That is why the source article emphasizes a line worth repeating: changing only the surrounding system design moved an agent dramatically up the leaderboard. As the MarkTechPost summary puts it, swapping only the surrounding setup moved performance from roughly 30th place to the top five on LangChain’s Terminal-Bench.
For builders, that is a useful corrective. Model selection still matters, but the coding agent harness now looks more like the product decision than the deployment footnote.
Why the harness matters more than the model choice
The strongest detail in this story is architectural, not promotional. The source separates the visible interface from the core loop, which makes it easier to reason about where quality actually comes from.
In interactive mode, the problem is not just generation quality. It is steering quality. If a user types while a tool call is still in flight, the turn can become inconsistent. Decode’s answer is to buffer that input and drain it only at safe boundaries, specifically MODEL_REQUEST and WOULD_STOP. That is a concrete example of harness engineering affecting reliability before the model has changed at all.
This same pattern shows up in other agent systems. Anthropic’s guidance on building effective agents and OpenAI’s agent design materials both point teams toward orchestration, tool control, and state handling rather than treating the model as the whole system.
That is also the point where implementation work becomes operational work. Teams moving from prototypes to production often need queueing, retries, observability, and process integration more than they need one more model bake-off. In that context, a natural next step is AI business process automation, especially when agent runs need to connect with ticketing, approval, or back-office workflows.
Why interactive agents need low-latency APIs
Interactive, online mode is the easiest to understand because the human feels every delay. A terminal UI is tied to one live in-memory session, and tokens stream back as they arrive. In this mode, time-to-first-token and responsiveness dominate everything else.
That is why the provider choice usually tilts toward hosted APIs. Per-token pricing can be expensive at scale, but when a developer is actively steering the session, cost per completed batch matters less than whether the system responds in a second or in ten. The source article’s operational detail is useful here: Enter, Alt+Enter, and Esc are not just UI conveniences. They map to different queue states and safe interruption points.
The trade-off is obvious. Interactive mode gives the highest control and the fastest human feedback loop, but it is also the worst place to leave expensive compute sitting idle. That matters even more when the model behind the coding session needs high-end GPU capacity.
How remote offline and async runs change the cost model
Remote, offline mode is different because nobody is watching each step. Here, an agent runtime can fan a backlog of tickets out across parallel workers, each producing its own pull request or output artifact. The source describes Decode using Kitaru from ZenML on GCP, with execution running inside Modal sandboxes.
That changes what good looks like. Resumability matters more than first-token latency. If a sandbox crashes halfway through a long task, the valuable feature is not chat smoothness; it is recovering from the last recorded step without restarting the whole run.
Async, online mode sits between chat and batch. The user starts the job while online, but the work moves to a queue and finishes in the background. This is where background agents become practical for Slack-triggered review, asynchronous code checks, or long-running PR preparation. The user experience feels live at the start, but the billing profile looks much closer to batch.
For many teams, this middle mode is the most underused. It preserves responsiveness at the handoff point without forcing the infrastructure to behave like a full interactive session the entire time.
The provider economics behind each mode
The source article’s arithmetic is what makes the story newsworthy rather than conceptual. It compares 1,000 documents at 30,000 input tokens each and about 500 output tokens per document. At frontier API prices of $3 per million input tokens and $15 per million output tokens, the total comes to about $97. Batched on a serverless GPU at around 3,000 tokens per second, the same workload lands under three hours of GPU time, or roughly $13.
That is not a marginal difference. It is a sign that provider selection should follow workload physics.
The reverse example is just as important. The course notes that Decode’s default test model, Qwen3.6 35B, can run on a single H200. Using Modal pricing, the article calculates H200 SXM at roughly $0.001261 per second, or about $4.54 per hour. If an interactive coding session sits idle for ten hours waiting on a human confirmation, that idle time alone costs about $45.
This is the practical rule:
- Interactive sessions are latency-bound, so token-priced hosted inference often makes sense.
- Remote and sandboxed agents are throughput-bound, so GPU-hour pricing gets more attractive as volume rises.
- Async work should be evaluated like queue-owned batch, not like chat.
There is also a capacity planning angle. Modal’s analysis of serverless GPU pricing argues that serverless wins when peak demand is much higher than average demand. The source cites typical reservation discounts of 2x to 5x, against peak-to-average ratios of 5x to 10x in agentic workloads. If utilization is uneven, reserved capacity can look efficient on paper and still lose in practice.
What this means for teams building coding agents
The practical takeaway is simple: choose the run mode before choosing the provider, and choose the provider before arguing over model fine margins. Teams building coding agents in 2026 are no longer deciding only on model quality. They are deciding whether the workflow is human-steered, queue-driven, or batch-oriented.
That also means different teams should standardize differently. Developer tools teams with live terminal use cases may accept higher per-token pricing for better responsiveness. AI infrastructure teams processing large job backlogs should bias toward throughput and resumability. Software teams in between should look closely at async patterns, because those often deliver the cleanest balance of user experience and spend.
The next thing to watch is whether benchmark culture catches up to this reality. If more evaluations separate core model quality from coding agent harness quality, teams may stop overpaying for the wrong layer. The more immediate watch item is vendor positioning: expect runtimes, sandboxes, and queue orchestration to become a bigger part of agent buying decisions over the next year.
Martin Kuvandzhiev
CEO and Founder of Encorp.io with expertise in AI and business transformation