by Vitaly NikitinHere is the failure mode that is harder to catch than an outage: everything looks fine and nothing works.
No alerts fire. Error rates are flat. The health dashboard is green. And every scheduled agent job that was queued behind a single misbehaving AI provider has silently stopped running.
That is what happened to us. One provider in the rotation started returning HTTP 402 — a billing or quota signal, not a transient network error. Our gateway treated it the way most gateways treat unexpected status codes: it propagated the failure up and stopped trying. Every turn queued behind that provider died the same way. The provider was not down. It was returning a valid HTTP response. The monitoring saw no errors in the usual sense.
We fixed it. The fix was smaller than the problem it solved. But the lesson behind it stretched further than we expected.
Why 402 is not the same as 500
When a provider returns HTTP 500, the intent is clear: something went wrong, try again. Most retry logic handles this correctly. HTTP 402 is different — it says “payment required,” which is a business-layer signal, not a transient fault. An AI provider might return 402 when a key has exceeded its quota, when a billing event hasn’t processed yet, or when a free tier has run out for the day.
The naive response is to treat 402 like 500 and retry. That burns quota faster, or spins on a limit that won’t lift for hours. The smarter response is to treat 402 as a signal to stop using this provider and try the next one — the same failover logic you’d apply to a provider that’s down, but triggered by a billing condition rather than a connectivity one.
That is what we shipped. When Forge’s AI gateway receives a 402 from a provider, it now steps down the preference list and routes to the next healthy provider — the same way it handles a provider that errors repeatedly on 5xx responses. The scheduled jobs that were queued behind the failing provider got to run. Nothing had to be retried by hand.
The technical surface area was small. The reasoning behind it took longer to get right.
The second problem: what the error said
While we were in there, we fixed something else.
When an AI model refuses a request on content-policy grounds, it returns a structured refusal — typically a clear message explaining that it won’t answer. That is not a system failure. It is the model making a decision. But our gateway was surfacing it as “tool budget exhausted” — a fabricated system error that told the user nothing true about what had happened.
A user watching an agent turn end with “tool budget exhausted” would reasonably conclude that a resource limit had been hit. Maybe they’d look at their usage dashboard. Maybe they’d file a support ticket. They would not conclude that the model had made a policy decision, because that is not what they were told.
This matters more than it might seem. When an agent gives you a false error message, it erodes trust in the infrastructure itself. You start doubting whether the platform is reliable, when the thing that actually happened was a model declined to answer. The infrastructure is working exactly as designed. The model is working exactly as designed. But the surface told you a lie.
The fix was to detect content-policy responses at the gateway layer and surface them as an honest in-thread decline — not a system error, not a retry candidate, but an accurate signal: “this model won’t answer this question.” Whether the user decides to rephrase, switch models, or accept the outcome, they’re making that decision with accurate information.
What these two things have in common
HTTP 402 failover and honest content-policy reporting are not obviously related. One is about routing; the other is about error messaging. But the underlying problem is the same: a system that appears to be working is actually broken in a way the surface doesn’t show you.
The industry term for this is “green but wrong” — a phrase that’s become shorthand for a class of AI reliability problems that observability tooling wasn’t designed to catch. Dev.to’s engineering roundup from this week names it as one of the dominant production AI failure modes in 2026: dashboards that show all green while agents are quietly returning bad results, silently degraded, or — in our case — not running at all.
Traditional observability watches for errors, latency, and throughput. “Green but wrong” failures don’t trigger any of those. They require a different kind of attention: understanding what the system is supposed to do and checking whether it is actually doing it, not just whether it is running without exceptions.
The compounding effect of unclear failures
There is a cost that accumulates when error messages lie. Not just the immediate confusion — the longer-term erosion of confidence in the system as a whole.
If “tool budget exhausted” appears when a model made a policy decision, and “tool budget exhausted” also appears when a real resource limit was hit, users start treating both the same way. They lose the ability to distinguish between “I need to adjust my prompt” and “I need to check my usage quota.” They stop trusting the error surface to tell them anything actionable.
We have seen this pattern play out in other systems too. Silent failures in CI pipelines. Zero-exit-code commands that don’t do what they claim. The diagnostic message that always says the same thing regardless of what actually went wrong. The common thread is a surface that has decoupled from the underlying reality — where the message you read is no longer a reliable signal of what happened.
The fix is always the same in principle: make the system say what it means. The mechanism varies. In the gateway, it meant classifying 402 as a failover trigger rather than a fatal error. It meant detecting policy refusals and reporting them honestly. Neither fix required rearchitecting anything. Both required deciding that accuracy in error reporting is not optional.
What this looks like in practice
Forge’s multi-provider routing has always handled 5xx errors and provider health degradation. It now handles 402 responses with the same logic: the request moves to the next provider in your preference order, transparently, without requiring a retry. Your agents don’t need to know which provider is having a billing moment; they just get an answer.
Content-policy refusals now appear as what they are — an honest decline from the model, in the conversation thread, with enough context to understand what happened. Not a system error. Not a retry loop. A clear signal.
Neither of these is a dramatic new capability. Both of them make the existing capability more trustworthy. That distinction matters to us: a system that runs reliably and tells you the truth about what it’s doing is more useful than a system with more features and less accuracy.
If you’re building agents that run unsupervised — scheduled jobs, overnight pipelines, multi-step workflows that need to finish without a human in the loop — the cost of silent failures is real. Every turn that dies with a misleading error is a turn someone has to investigate manually, often hours after it happened. Every queued job that stops with no signal is work that didn’t get done.
The goal is not zero failures. It’s zero invisible failures. When something goes wrong, the system should say so clearly. When something can be recovered automatically, it should be. And when a model makes a decision rather than hitting an error, the surface should call it what it is.
That is what we shipped. The scheduled jobs run. The error messages tell the truth.
Forge’s multi-provider routing handles failover, health tracking, and provider preference across all connected AI providers. See how it works →