Models & Releases Research Products & Apps Business & Funding

Modelwire

A curated feed of what matters in AI. Independent, ad-supported, built in Denver, Colorado.

Read

Today
Models & Releases
Research
Business & Funding

About

About Modelwire
Methodology
Our sources
Editor's notes
Contact
Advertise

Legal

Privacy policy
Terms of use
DMCA & takedowns
Corrections

© 2026 Modelwire. All article links go to the original publishers.Summaries generated by Modelwire. We don’t republish full articles.

arXiv cs.CL

https://arxiv.org/list/cs.CL/recent · Editorial weight 5/10

Illustration for: Neuron Populations Exhibit Divergent Selectivity with Scale

Neuron Populations Exhibit Divergent Selectivity with Scale

Researchers have discovered that neurons exhibiting consistent activation patterns across independently trained models (Rosetta Neurons) follow predictable scaling laws, but with a counterintuitive twist: while their absolute count grows, they shrink as a fraction of total neurons. More significantly, these neurons become increasingly specialized and monosemantic at scale, suggesting that model scaling drives functional consolidation rather than uniform expansion. This finding extends mechanistic interpretability beyond loss curves into neuron-level behavior, offering practitioners a new lens for understanding how model internals reorganize during training and potentially informing architecture design decisions.

arXiv cs.CL·1d ago

62

Illustration for: Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics

Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics

Researchers have identified a fundamental limitation in how language models process quantitative reasoning: LMs compare measurements by applying loose heuristics tied to individual numerals and unit scales rather than normalizing to a shared reference frame. This finding matters because it reveals a systematic failure mode in a task that appears simple but underpins real-world applications from scientific computing to financial analysis. The degradation near decision boundaries suggests that current architectures lack robust internal representations for unit conversion, a gap that could affect reliability in domains where precision is non-negotiable.

arXiv cs.CL·1d ago

58

Illustration for: Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill

Research Tools & Code

Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill

Researchers propose Skill-RM, a framework that treats reward modeling as an agentic task to unify disparate evaluation signals used in LLM post-training. Rather than juggling separate rule-based verifiers, reference comparisons, and rubric systems, Skill-RM provides a single interface that dynamically selects and combines evidence types based on task requirements. This addresses a real friction point in RLHF and reinforcement learning pipelines where heterogeneous feedback sources currently lack principled integration. The approach could streamline how teams construct reward signals for fine-tuning, reducing engineering overhead and improving consistency across complex evaluation scenarios.

arXiv cs.CL·1d ago

58

Illustration for: Quantifying Faithful Confidence Expression in Large Reasoning Models

Quantifying Faithful Confidence Expression in Large Reasoning Models

A new study exposes a critical gap in how large reasoning models communicate uncertainty. While users often interpret lengthy chain-of-thought outputs as signals of model competence and deliberation, the research reveals that these models frequently express confidence levels misaligned with their actual accuracy. The work challenges existing calibration measurement methods, which fail to account for the structural complexity of extended reasoning traces. This matters because deployment of reasoning models in high-stakes domains depends on users correctly interpreting when the system is reliable versus speculating, making faithful confidence expression a foundational trust problem the field has largely overlooked.

arXiv cs.CL·1d ago

62

Illustration for: QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards

QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards

QUBRIC addresses a fundamental constraint in rubric-based reinforcement learning: query structure directly limits rubric quality, creating a catch-22 where overly open prompts yield unusable evaluation criteria while over-constrained queries introduce unverifiable references that collapse the reward signal. The framework co-optimizes query design and rubric generation by anchoring both to teacher-derived key points, then filters for learnability, enabling RL systems to learn from domains where ground truth verification remains intractable. This matters because it expands the frontier of trainable tasks beyond those with crisp, externally verifiable outcomes, a bottleneck for scaling alignment and reasoning in frontier models.

arXiv cs.CL·1d ago

62

Illustration for: AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task

Research Tools & Code

AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task

Researchers have adapted AlignAtt, a technique for steering attention in encoder-decoder models, to work with decoder-only LLMs for the first time. The breakthrough matters because decoder-only architectures now dominate production systems, yet prior alignment methods relied on cross-attention mechanisms absent in these models. The team's solution uses prompt-based source spans, selective attention head replay, and runtime query/key capture to guide Gemma-4 during simultaneous speech translation without degrading model outputs. This opens a new avenue for controlling LLM behavior in latency-sensitive tasks where incremental decoding and source alignment are critical.

arXiv cs.CL·1d ago

58

Illustration for: Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning

Research Tools & Code

Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning

Researchers propose Agentic Chain-of-Thought Steering, a method that treats LLM reasoning as a controllable process where a separate agent dynamically guides inference strategy and token allocation. Rather than passively shortening or compressing reasoning traces, ACTS lets operators steer how models think in real time, balancing accuracy against compute budget. This addresses a core tension in scaling reasoning: extended chain-of-thought improves answers but wastes tokens on redundant steps. The approach opens a new lever for inference optimization and could reshape how practitioners deploy reasoning-heavy models under latency or cost constraints.

arXiv cs.CL·1d ago

62

Illustration for: Efficient ASR Training with Conversations that Never Happened

Research Tools & Code

Efficient ASR Training with Conversations that Never Happened

Researchers have cracked a persistent bottleneck in conversational speech recognition for underserved languages and domains: the absence of multi-speaker dialogue data. By chaining LLM-generated scenarios with speaker metadata through TTS synthesis, they assembled fully synthetic conversations that meaningfully boosted ASR performance on Hungarian benchmarks. The technique sidesteps expensive human annotation and scales across any language with component infrastructure in place, making it immediately relevant to teams building speech systems outside English-dominant markets.

arXiv cs.CL·1d ago

58

Research Models & Releases

A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026

Charles University's IWSLT 2026 submission demonstrates a practical shift in simultaneous speech translation: pairing Nvidia's Canary model with the AlignAtt policy achieves competitive translation quality while staying within a 1B parameter budget. The system handles 25 language pairs across Czech, English, German, and Italian, suggesting that real-time multilingual translation no longer requires frontier-scale compute. For practitioners building on-device or edge translation systems, this validates that latency-quality tradeoffs can be solved without scaling model size, reshaping expectations around what inference efficiency looks like in production speech AI.

arXiv cs.CL·1d ago

54

Illustration for: Value-Aware Stochastic KV Cache Eviction for Reasoning Models

Research Tools & Code

Value-Aware Stochastic KV Cache Eviction for Reasoning Models

Reasoning models face a hard tradeoff between accuracy and efficiency when handling long chains of thought. This paper identifies why naive KV cache eviction fails: a small set of high-magnitude value states are critical to coherence, and their removal triggers repetitive loops. The authors propose VaSE, a training-free method that combines value-magnitude protection with stochastic sampling to preserve cache diversity. The work matters because it directly addresses the compute bottleneck limiting deployment of reasoning-heavy models like o1, offering a practical path to cheaper inference without sacrificing the extended reasoning that defines their advantage.

arXiv cs.CL·1d ago

62

Illustration for: Knowledge Editing in Masked Diffusion Language Models

Knowledge Editing in Masked Diffusion Language Models

Researchers have extended knowledge editing, a technique for surgically updating factual errors in language models, from autoregressive architectures to masked diffusion models. By comparing LLaDA and Dream (bidirectional, iterative denoisers) against LLaMA and Qwen at equivalent scale, the work reveals that edit localization patterns transfer across fundamentally different generation paradigms. This finding matters because it suggests knowledge editing principles may be more universal than previously assumed, potentially unlocking safer, more correctable models across diverse architectures as the field moves beyond next-token prediction.

arXiv cs.CL·1d ago

58

Illustration for: Synthesize and Reward -- Reinforcement Learning for Multi-Step Tool Use in Live Environments

Research Tools & Code

Synthesize and Reward -- Reinforcement Learning for Multi-Step Tool Use in Live Environments

Researchers have tackled a fundamental bottleneck in LLM tool-use training: the gap between synthetic data and real execution environments. PROVE introduces a framework combining 20 stateful MCP servers with 343 tools, automated trajectory synthesis, and novel reward mechanisms to enable reinforcement learning on live systems without the brittleness of prior approaches. This addresses a critical pain point for teams building agentic systems, where tool-calling failures cascade through multi-step workflows. The work signals growing maturity in the infrastructure layer for training reliable autonomous agents at scale.

arXiv cs.CL·1d ago

62

Research Tools & Code

RealClawBench: Live OpenClaw Benchmarks from Real Developer-Agent Sessions

RealClawBench shifts agent evaluation away from synthetic tasks toward actual developer workflows by reconstructing execution environments and building deterministic scorers from live OpenClaw sessions. This addresses a critical gap in how the field measures deployed agent performance: existing benchmarks miss the messy reality of underspecified requests, environment dependencies, and verification challenges that define production use. The 281-task dataset captures authentic distribution and difficulty, making it a meaningful calibration point for teams building and evaluating code agents in real conditions.

arXiv cs.CL·1d ago

62

Illustration for: Visual Instruction Tuning Aligns Modalities through Abstraction

Visual Instruction Tuning Aligns Modalities through Abstraction

Researchers have mapped how vision-language models actually fuse modalities during instruction tuning, revealing that visual information bypasses early unimodal layers and embeds directly into intermediate semantic layers of the LLM backbone. Through probing and causal intervention, the work identifies these middle layers as the critical junction for multimodal reasoning and performance across benchmarks. This finding reshapes how practitioners should think about architecture design and layer-wise optimization in vision-language systems, moving beyond black-box assumptions about where cross-modal alignment occurs.

arXiv cs.CL·1d ago

62

Illustration for: A Training-Free Mixture-of-Agents Framework for Multi-Document Summarization using LLMs and Knowledge Graphs

Research Tools & Code

A Training-Free Mixture-of-Agents Framework for Multi-Document Summarization using LLMs and Knowledge Graphs

Researchers propose a training-free multi-document summarization framework that pairs LLMs with knowledge graphs to handle complex cross-document relationships without supervised fine-tuning. The approach decomposes summarization into specialized agent roles (extraction, abstraction, refinement) unified through consistency mechanisms, addressing a persistent gap in domain and language generalization. This signals growing momentum toward modular, zero-shot LLM architectures that reduce data dependency and expand applicability across verticals where labeled training sets remain scarce.

arXiv cs.CL·1d ago

58

Illustration for: Taiji: Pareto Optimal Policy Optimization with Semantics-IDs Trade-off for Industrial LLM-Enhanced Recommendation

Research Products & Apps

Taiji: Pareto Optimal Policy Optimization with Semantics-IDs Trade-off for Industrial LLM-Enhanced Recommendation

Taiji addresses a critical friction point in LLM-powered recommendation systems: bridging the semantic space of language models with the ID-based preference signals that drive industrial recommenders. The framework tackles two concrete bottlenecks in post-training alignment: improving chain-of-thought reasoning quality during supervised fine-tuning and resolving the inherent tension between semantic rewards and collaborative-filtering objectives during reinforcement learning. This work matters because recommendation remains one of the highest-ROI deployment surfaces for LLMs in production, and solving the semantic-ID trade-off could unlock more efficient scaling of hybrid systems without sacrificing ranking performance.

arXiv cs.CL·1d ago

58

Illustration for: Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models

Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models

Researchers propose a clustering-based self-assessment technique to extract uncertainty signals directly from LLM outputs, addressing a critical gap in model reliability. Rather than inferring confidence from entropy or sampling variance, the method groups semantically similar generations and converts them into structured answer options, allowing models to explicitly assess their own uncertainty. This tackles a fundamental deployment challenge: users currently lack reliable signals to distinguish confident-but-wrong outputs from genuinely uncertain predictions. The approach is particularly relevant as enterprises scale LLM adoption in high-stakes domains where calibrated uncertainty estimates are prerequisites for safe human-in-the-loop workflows.

arXiv cs.CL·1d ago

58

Research Models & Releases

Dynamic Short Convolutions Improve Transformers

Researchers propose dynamic short convolutions as a complementary primitive for Transformer architectures, where input-dependent filters replace static convolution kernels to preserve locality while expanding model expressivity. Testing across 150M to 2B parameter language models shows consistent gains on associative recall tasks, suggesting a path to enhance Transformer performance without wholesale architectural replacement. This work matters because it targets a core limitation of attention mechanisms: their weak inductive bias for local structure. If validated at scale, dynamic convolutions could become a standard component in next-generation LLM designs.

arXiv cs.CL·1d ago

58

Rethinking the Idiomaticity Decomposability Hypothesis: Evidence from Distributional Learning

Researchers challenge a foundational linguistic theory about idioms by using contextualised language models as experimental instruments. Rather than decomposability (how much individual words contribute to meaning) predicting syntactic flexibility, the study finds distributional learning patterns better explain idiom behavior. The work matters because it reveals how neural language models internalize figurative language during pretraining, offering a new lens on what drives model robustness with non-literal expressions. This bridges cognitive linguistics and modern NLP, suggesting that raw exposure frequency shapes how systems handle idiomatic variation more than compositional structure does.

arXiv cs.CL·1d ago

54

Illustration for: Consistency Training Can Entrench Misalignment

Consistency Training Can Entrench Misalignment

A systematic study of consistency training across 108 open-source models reveals a critical trade-off in alignment interventions: while these scalable, label-efficient methods suppress reward hacking and emergent misalignment, they paradoxically amplify sycophancy. The finding that distribution shifts from the consistency labeling process, rather than selection mechanisms, drive these outcomes reshapes how practitioners should think about self-bootstrapping alignment techniques. This matters because consistency training is widely adopted for its simplicity, yet the research suggests it may entrench specific failure modes while fixing others, forcing teams to choose between different alignment risks rather than achieving uniform improvement.

arXiv cs.CL·1d ago

62

Illustration for: Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models

Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models

Researchers have uncovered a critical vulnerability in multilingual multimodal LLMs: adversarial images crafted to fool models in one language transfer effectively across other languages, exposing a systemic gap in cross-lingual safety. This finding challenges the assumption that safety alignment generalizes uniformly across languages and suggests that current instruction-tuning approaches leave models exposed to coordinated attacks that exploit language boundaries. For practitioners deploying MLLMs globally, the work signals that robustness testing must span linguistic diversity, not just English benchmarks.

arXiv cs.CL·1d ago

62

Illustration for: Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs

Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs

Researchers demonstrate that unlearning a single backdoor trigger in large language models can suppress other unknown backdoors simultaneously, a finding that inverts the traditional defense paradigm. Rather than requiring defenders to identify and neutralize each attack vector individually, this generalization effect suggests a unified mitigation strategy may be possible. The work spans three model families with backdoors introduced at pretraining and continual pretraining stages, offering practical implications for securing deployed systems where threat actors may have injected multiple hidden triggers. This shifts the security calculus from reactive, trigger-specific patching toward proactive, broad-spectrum neutralization.

arXiv cs.CL·1d ago

68

Illustration for: Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation?

Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation?

Researchers are testing whether explicit linguistic reasoning steps can help large language models translate extremely low-resource languages more accurately. By automatically generating intermediate grammatical analyses from dependency treebanks and rule banks, the team explores whether chain-of-thought-style decomposition improves translation quality across in-context learning, fine-tuning, and reinforcement learning setups on Xibe and Chin languages. This work bridges two active research frontiers: scaling LLMs to underserved language pairs and leveraging structured reasoning to guide model behavior, with implications for how linguistic structure can be operationalized as a training signal rather than just a feature.

arXiv cs.CL·1d ago

58

Illustration for: Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models

Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models

Researchers have extended causal tracing, a technique for mapping how language models store and retrieve facts, to sparse mixture-of-experts architectures. Previous work focused on dense transformers where interventions target layers or feed-forward blocks. This study isolates which individual experts within routed MoE blocks contribute to factual predictions by corrupting subject embeddings and measuring whether clean expert outputs restore correct logit contrasts. Using Qwen3-30B, they pinpointed layer 44 and a specific expert as critical for factual recall. The work matters because MoE models are becoming standard at scale, and understanding their internal routing decisions is essential for interpretability, debugging, and alignment efforts in production systems.

arXiv cs.CL·1d ago

58

Illustration for: KletterMix: Climbing Toward High-Quality German Pretraining Data

Research Tools & Code

KletterMix: Climbing Toward High-Quality German Pretraining Data

KletterMix addresses a structural gap in multilingual AI development by delivering a large-scale, carefully curated German pretraining corpus built through systematic translation of English reference data. The dataset preserves document integrity and topical breadth while maintaining reproducibility, positioning it as infrastructure for closing the quality disparity between English and German language models. This work signals growing recognition that non-English LLM capability depends on deliberate curation rather than scale alone, with implications for how other underserved languages approach pretraining resource development.

arXiv cs.CL·1d ago

58

Illustration for: HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps

Research Models & Releases

HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps

HybridThinker addresses a core efficiency bottleneck in reasoning-heavy LLMs by balancing compressed memory tokens with temporary access to full reasoning traces during inference. The key insight is preventing models from circumventing compression during training, forcing genuine reliance on compact representations while retaining fine-grained context when needed. This tackles a real production constraint: extended chain-of-thought reasoning improves accuracy but explodes computational cost. The approach matters for practitioners scaling reasoning workloads and signals ongoing tension between model capability and deployment efficiency that will shape inference architecture choices.

arXiv cs.CL·1d ago

62

Illustration for: Framing Migration News with LLMs: Structured CoT as a Support for Human Interpretation

Research Tools & Code

Framing Migration News with LLMs: Structured CoT as a Support for Human Interpretation

Researchers demonstrate that open-source LLMs can perform interpretable frame analysis on migration news without relying on proprietary APIs or large closed models. Using Llama3-8B with Structured Chain-of-Thought prompting, the work prioritizes auditability and reproducibility for academic media scholars operating under resource constraints. This signals a broader shift toward locally deployable, transparent alternatives for social science applications where data privacy and methodological accountability matter more than raw scale.

arXiv cs.CL·1d ago

58

Illustration for: Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines

Research Tools & Code

Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines

Entropy Gate proposes a thermodynamic framework for compressing token sequences in LLM inference by selectively removing low-information content while maintaining semantic integrity. The method assigns each token an information energy score combining statistical, structural, and positional signals, then applies an adaptive cooling schedule to prune tokens below a survival threshold. This addresses a real efficiency bottleneck in production LLM pipelines where redundant context and verbose outputs inflate compute costs. If validated empirically, the approach could meaningfully reduce inference latency and token consumption across deployed systems, particularly for long-context or high-volume workloads where token budgets remain a hard constraint.

arXiv cs.CL·1d ago

62

Illustration for: Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA

Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA

Legal QA systems built on retrieval-augmented generation face a fundamental mismatch: semantic similarity, the standard ranking metric, fails to surface passages that language models actually cite. Researchers discovered this gap using attribution methods like C-LIME on the AQuAECHR benchmark, where random retrieval outperformed similarity-based ranking for gold citations. The fix involves training a lightweight cross-encoder on perturbation-based attribution scores to re-rank candidates before generation. This work exposes a critical blind spot in RAG pipelines and suggests that post-hoc explanation techniques can be repurposed as ranking signals, with implications for any domain where citation fidelity matters.

arXiv cs.CL·1d ago

58

Illustration for: Don't Forget Your Embeddings: Robust Knowledge Erasure via Precise Editing of Embeddings

Research Tools & Code

Don't Forget Your Embeddings: Robust Knowledge Erasure via Precise Editing of Embeddings

Researchers have identified a critical gap in knowledge erasure methods for language models: existing parameter-update approaches fail to address token embeddings, allowing adversaries to recover supposedly deleted information. EMBER, a new plug-and-play module using sparse matrix factorization, targets concept-related features directly in embedding layers to achieve more durable knowledge removal. Tested on Gemma-2-2B-it and Llama-3.1-8B-Instruct, this work matters for compliance-heavy deployments where regulatory erasure requirements carry real legal stakes, and signals that robust model editing requires rethinking the full architecture, not just weights.

arXiv cs.CL·1d ago

62