286,522 Input Tokens Down to 196,983: A Team Cut Its AI Agent's Token Bill About 31% and Published What Worked and What Broke
- Level Advanced
- Setup1–3 hrs (estimated)
- Evidence Some evidence
- CostNot stated in source
- No-codeNo
What they did
A team running an AI business intelligence agent found it re-read 17,000 tokens of instructions and tool manuals before every thought. They measured first with an open-source bench and quality checks, then made cuts in order: removing 10 tools the agent never called, shortening old tool results but never the current turn, ordering prompts so they cache, and adding a grouped query so the model asks once instead of five times. That last change cut one real question by 41%. A three-question conversation dropped from 286,522 to 196,983 input tokens, and they also list what failed.
How it went
dev.to field report by the developer (Oct 28, 2026): first call 17,070 tokens before the question; 36 tools cut to 26 (first call 17,070 to 13,598 on the bench); old tool results shortened 21-26%; cache-friendly prefix 11% to 98% (benchmark upper bound); grouped query 71,617 to 42,130 tokens (-41.2%) on a real run; three-question conversation 286,522 to 196,983 (-31.3%) offline; bench overestimated real gains by 9-16%. Open-source bench
Worth knowing
- dev.to field report by the developer (Oct 28, 2026): first call 17,070 tokens before the question; 36 tools cut to 26 (first call 17,070 to 13,598 on the bench); old tool results shortened 21-26%; cache-friendly prefix 11% to 98% (benchmark upper bound); grouped query 71,617 to 42,130 tokens (-41.2%) on a real run; three-question conversation 286,522 to 196,983 (-31.3%) offline; bench overestimated real gains by 9-16%. Open-source bench.
Prompt
Cleaned version rewritten from the source — not a direct quote.
Help me cut the input tokens my agent uses without hurting its answers. First, build a small benchmark: [5-10] fixed scenarios from real use, with a quality check for each (the facts the answer must contain). Measure input tokens per model call and per scenario as a baseline. Then test these cuts one at a time and report the token change and whether every quality check still passes: remove tools the agent never calls, shorten tool results from past turns only (never the current turn), make the prompt prefix stable so it caches, and find places where one combined tool call could replace several. Reject any cut that fails a check. My agent's code is in [path].
Recommended
Try it in Grok Bot
- Copy the prompt (button below).
- Open Grok Bot chat and paste it. Replace any [bracketed] placeholders.
- Connect the listed tools (as needed) as connectors when Grok Bot asks.
- If the task repeats, ask: "Make this a daily routine."
Alternatives: the same prompt works in Claude, ChatGPT Agent, OpenClaw.
Set it up
Stack: Custom Agent
- Read the original source for full context.
- Connect the tools your agent needs.
- Paste the prompt and replace [bracketed] placeholders.
- Run once manually and check the output before scheduling it.
Generic suggested steps, not from the source.
Evidence
dev.to field report by the developer (Oct 28, 2026): first call 17,070 tokens before the question; 36 tools cut to 26 (first call 17,070 to 13,598 on the bench); old tool results shortened 21-26%; cache-friendly prefix 11% to 98% (benchmark upper bound); grouped query 71,617 to 42,130 tokens (-41.2%) on a real run; three-question conversation 286,522 to 196,983 (-31.3%) offline; bench overestimated real gains by 9-16%. Open-source bench
Source: Web · Verified by us: not yet · Report broken or outdated