
ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues
ReproRepo addresses a critical friction point in AI research: the gap between published results and reproducible code. By mining GitHub issues as natural labels for real-world reproduction failures, researchers have built a scalable evaluation framework that sidesteps manual curation bottlenecks. Testing four frontier LLM agents on 1,149 ML papers reveals that even non-executing models can surface genuine blockers, suggesting a path toward automated reproducibility auditing. This matters because reproducibility remains a bottleneck for both research velocity and trust in published claims, and agent-assisted diagnosis could accelerate debugging cycles across the field.62



























