DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition

The core insight is that LLM annotation errors in information extraction tend to be systematic rather than random, meaning a single model will repeat the same mistakes consistently. DiZiNER's approach of simulating the friction between multiple annotators is designed to surface and correct those blind spots before they propagate into labeled data.

This connects directly to the CoopEval paper covered yesterday, which found that LLM agents in multi-agent settings default to self-interested behavior rather than productive coordination. DiZiNER is essentially betting on the opposite dynamic: that structured disagreement between LLM agents produces better collective output than any single model alone. Both papers are probing the same underlying question about whether multi-LLM architectures can compensate for individual model weaknesses. The DiscoTrace work from the same day is also relevant, since it documented how LLMs systematically favor certain rhetorical patterns over others, which is precisely the kind of systematic bias DiZiNER is trying to counteract in the annotation context.

The real test is whether DiZiNER's gains hold on entity types that are genuinely underrepresented in LLM pretraining data, not just on standard CoNLL-style benchmarks. If follow-up evaluations on low-resource or domain-specific NER corpora show similar improvements, the disagreement-resolution mechanism is doing real work rather than exploiting familiar entity distributions.