Have your agent record video demos of its work with shot-scraper video
Simon Willison's shot-scraper 1.10 introduces video recording capabilities for web automation workflows, addressing a concrete gap in AI agent evaluation. The tool lets developers capture visual proof that autonomous agents complete tasks correctly, moving beyond logs and text output. This matters because LLM-powered agents increasingly need to demonstrate competence to stakeholders, and having standardized tooling for video evidence reduces friction in adoption and debugging. The storyboard-driven approach makes it accessible for non-specialists to script complex interaction sequences.
Modelwire context
ExplainerThe deeper point is not the video recording itself but what it reveals about the current state of agent evaluation: most developers are still relying on logs and return values to verify autonomous behavior, which means errors that are visually obvious can go undetected for entire workflow runs.
This is largely disconnected from recent Modelwire coverage. The closest thematic thread is the .gov AI design story from June 30, which documented how AI-generated outputs failed to meet real-world requirements precisely because there was no adequate review loop catching visual and usability problems before they shipped. Shot-scraper video is a small-scale, developer-facing answer to that same class of problem: building a feedback mechanism that surfaces what text output cannot. The .gov case involved institutional failure at scale; this tool addresses the same verification gap at the individual workflow level. The connection is structural rather than direct.
Watch whether CI platforms like GitHub Actions or testing frameworks like Playwright's own tooling absorb video-capture patterns from shot-scraper within the next two release cycles. If they do, it signals the gap Willison identified is widely recognized; if not, this stays a niche utility for solo developers.
Coverage we drew on
- Trump's plan to redesign every .gov website leads to AI-designed horrors · Ars Technica - AI
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsSimon Willison · shot-scraper · Playwright · Datasette
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. The full content lives on simonwillison.net. If you’re a publisher and want a different summarization policy for your work, see our takedown page.