DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching

The more pointed finding here is not just that LLMs give worse coaching, but that prompting alone cannot close the gap. Even when models are explicitly instructed to behave like human coaches, they fail to anchor guidance in what is visually present on screen, suggesting the deficit is architectural rather than a prompt engineering problem.

This connects directly to the 'Self-Study Reconsidered' paper from the same day, which showed that synthetic QA generation introduces systematic biases in what models learn to attend to. DigitalCoach extends that concern into a different domain: if models are trained on data that underrepresents explanation, verification, and error diagnosis, no amount of instruction-following fine-tuning will surface behaviors that were never in the training signal. Both papers are pointing at the same upstream problem from different angles. The visual grounding failure also echoes the surrogate fidelity work, which found that surface-level behavioral alignment between models can mask deep internal divergence, here the surface command looks correct but the reasoning scaffolding is absent.

Watch whether any lab releases a coaching-specific fine-tune trained on DigitalCoach or a comparable instructional dataset within the next six months. If benchmark performance on visual grounding tasks improves but coaching quality on held-out sessions does not, that confirms the deficit requires new training objectives rather than better base models.