llm-echo adds reasoning and tool-call token accounting
Source published ·Modelwire updated
Original coverage: Simon Willison ↗·How Modelwire adds context
The development
Simon Willison updated llm-echo, a debug plugin for the LLM CLI tool that simulates model behavior without calling an actual LLM. The 0.5 release refines token accounting to reflect modern LLM complexity: tool calls, system prompts, and reasoning tokens now factor into simulated usage metrics. This matters for developers testing LLM integrations locally and for anyone building tooling around token-aware cost estimation. The update reflects how token accounting has evolved beyond simple prompt/completion splits as models gain reasoning and tool-use capabilities.
Modelwire’s AI-generated summary of coverage from Simon Willison.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The 0.5 release doesn't just add new token types to count; it signals that local testing infrastructure now needs to mirror the full complexity of production LLM behavior. Developers can no longer assume a simple prompt-in, completion-out model when reasoning tokens and tool calls are reshaping how costs and latency actually accrue.
This fits alongside the dependency fragility exposed in the llm-tools-datasette patch from yesterday. Both reveal how the LLM plugin ecosystem is maturing past toy examples into tooling that tracks real production constraints. Where that story showed brittleness from implicit dependencies, this one shows maturation in how plugins account for actual system behavior. Token accounting precision matters because, as the medical device determinism study showed, production deployments require auditability; developers testing locally need their simulations to reflect what they'll actually see in cost reports and latency profiles.
If Simon Willison or other LLM CLI maintainers add reasoning token simulation to llm-echo within the next two releases, that confirms reasoning is becoming a standard concern for local testing workflows. If it stays absent, it suggests reasoning tokens remain edge-case enough that most developers still test without them.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·Simon Willison
LLM plugin ecosystem exposed by HTTP client migration
Simon Willison patched llm-tools-datasette after a dependency shift in LLM 0.33 broke plugin compatibility. The core issue: migrating LLM's HTTP client to httpx2 exposed plugins that relied on implicit transitive dependencies rather than declaring their own. This highlights a structural fragility in the plugin ecosystem around LLM, Datasette's AI-native data tool. For developers building on…
MentionsSimon Willison · llm-echo · LLM · Datasette
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. Simon Willison originally reported this story as “llm-echo 0.5”. The full content lives on simonwillison.net. If you’re a publisher and want a different summarization policy for your work, see our takedown page.