Multimodal Evaluator Preference Collapse: Cross-Modal Contagion in Self-Evolving Agents
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
Self-evaluating AI agents suffer from systematic preference collapse when using language models as judges, but multimodal systems amplify this failure dramatically. Researchers found that GPT-4o evaluating DeepSeek-chat across text and vision tasks caused a single strategy to dominate 48% of selections, triple the text-only baseline. More critically, they identified cross-modal contagion: biases learned in one modality poison strategy selection in another. This reveals a fundamental fragility in self-improving agent loops that scales with model complexity, forcing practitioners to rethink feedback architectures for multimodal systems.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The more alarming finding isn't the collapse rate itself but the directionality: biases don't stay contained within the modality where they form. A flawed preference pattern learned while judging image tasks can corrupt text-task selection, meaning you can't isolate and fix modalities independently.
This connects directly to the misinformation propagation paper covered the same day ('Misinformation Propagation in Benign Multi-Agent Systems'), which found that false premises introduced by a single agent persist through group reasoning rather than getting corrected by debate. Both papers are describing the same underlying dynamic at different levels: feedback loops in multi-agent and self-evaluating systems amplify errors rather than damping them. The collective skill-building work in 'OpenClaw-Skill' is also relevant here, because any framework that iteratively constructs agent capabilities from self-assessed outputs inherits exactly this fragility. If the evaluator is collapsing preferences, the skill tree being built on top of those evaluations is compromised from the root.
Watch whether teams building production multimodal agent pipelines begin publishing ablations that separate evaluator modality from task modality. If that experimental design becomes standard in benchmark reporting within the next two conference cycles, this paper will have shifted how the field operationalizes evaluation hygiene.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsGPT-4o · DeepSeek-chat · Evaluator Preference Collapse · cross-modal contagion
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.