router overhead — rolling p95: 11ms. Illustrative model, not measured results.
Three ways in, ordered by how much of your stack you have to touch. The one we suggest starting with touches none of it and changes no answer your users see.
A
Your existing client library keeps its call signatures; requests go to Switchyard and come back in the shape they already came back in. The endpoint name is what ties a call to a ladder, a floor and a budget, so it is the one argument you have to add.
B
The gateway speaks the same wire format your application already speaks. Endpoints are derived from the path, which is coarser than naming them yourself, but it gets a service you cannot rebuild onto a ladder this week.
C
Requests are copied, fingerprinted and scored against candidate ladders offline. Your users keep getting answers from your current setup. At the end you have your own traffic’s difficulty distribution and a modelled bill instead of ours — which is the honest way to find out whether any of this applies to you.
Start with C. It is the only one of the three that cannot hurt you, and it is the one that produces an argument.
Five hops, and the first one never leaves your side. The fingerprint is computed in your process or your VPC, which means the request text is scored without being shipped anywhere to be scored.
| Hop | Runs | What lands on disk |
|---|---|---|
| Request | Your application | Nothing of ours. |
| Fingerprint | Your process or your VPC | Five axis scores and D. Not the text. |
| Decision | Switchyard | Chosen tier, the path through the tree, the reason for each edge. |
| Upstream call | Your provider | Token counts, latency per rung, errors. Bodies pass through. |
| Judge sampling | Switchyard, 6% of calls | A score and its basis. |
Stored by default
Not stored by default
Body retention is opt-in, per endpoint, with a window you set. Off is the default and stays the default until you turn it on — a routing layer that quietly accumulated everyone’s prompts would be a liability wearing a product’s clothes.
Design targets for this illustrative deployment, stated so they can be measured and argued with. They are not service commitments, and there is no SLA on this site.
| Metric | Target |
|---|---|
| Routing availability | 99.95% |
| Added overhead, p95 | ≤ 15 ms |
| Brownout detection | ≤ 60 s |
Four steps, each with a switch that puts you back where you started.
| Step | What happens | Rollback |
|---|---|---|
| Weeks 1–2 | Shadow only. Your traffic’s difficulty distribution and a modelled bill, nothing routed. | Stop mirroring. |
| Week 3 | One endpoint at 5%. Pick the one with the most routine traffic, not the most expensive one. | Set the share to zero. Takes effect on the next request. |
| Weeks 4–7 | Ramp that endpoint 25% → 100%, then the next endpoint. One at a time. | Per-endpoint share, same switch. |
| Ongoing | New tiers enter through the shadow and the swap window, never straight into production. | Promotion is reversible at every ramp step. |