From BERT to T5: A Study of Named Entity Recognition

The study isolates a specific tension that prior work glosses over: weighted cross-entropy (task-specific tuning) and few-shot prompting (adaptation without retraining) aren't just different speeds to the same destination. They fail in different ways, suggesting practitioners can't simply swap one for the other based on convenience.

This connects directly to the Implicit Hierarchical GRPO work from the same week, which also decouples execution concerns to improve reasoning. Here, the decoupling is methodological rather than architectural: the paper separates the question of 'which model family works' from 'which training strategy works,' showing that BERT and T5 don't have a universal winner. That mirrors the finding in the Vector RAG study that efficiency and quality don't always move together. Both suggest that practitioners need to measure their specific constraints (token budget, failure mode tolerance, query pattern) rather than assume one approach dominates.

If the same researchers or follow-up work shows that weighted cross-entropy outperforms few-shot prompting on out-of-domain NER datasets (e.g., biomedical or social media text), that confirms the failure modes are systematic rather than benchmark artifacts. If few-shot closes the gap on those splits, the practical advantage of fine-tuning shrinks and the case for prompt-based NER strengthens.