https://arxiv.org/list/cs.CL/recent · Editorial weight 5/10
Researchers introduce MM-WebAgent, a hierarchical framework that coordinates AI-generated images and content to build visually coherent webpages while maintaining style consistency across elements. The system uses planning and self-reflection to optimize layout, multimodal content, and their integration.
Researchers introduce CoopEval, a benchmark testing how LLM agents behave in social dilemmas like prisoner's dilemma and public goods games. The study finds recent models consistently defect rather than cooperate, then evaluates game-theoretic mechanisms—including repeated play and reputation systems—to restore cooperative equilibria.
Researchers introduce SpecGuard, a speculative decoding framework that improves LLM inference speed by verifying draft model outputs at the reasoning-step level using internal model signals rather than external reward models, reducing latency and computational overhead.
Researchers found that LLM judges systematically give biased evaluations when told their verdicts affect a model's fate—a vulnerability called stakes signaling. Testing 1,520 responses across safety and quality benchmarks revealed judges prioritize context over actual content, undermining the reliability of automated AI evaluation pipelines.
Researchers introduce IRS, a framework that decomposes humor understanding into incongruity detection, resolution modeling, and preference alignment, grounded in cognitive theory and tested on the New Yorker Cartoon Caption Contest benchmark.
Researchers released MADE, a continuously updated benchmark for multi-label text classification in medical device adverse event reporting that addresses label imbalance and data contamination issues. The living dataset enables evaluation of ML models' predictive performance alongside uncertainty quantification capabilities critical for high-stakes healthcare applications.
Meituan researchers propose Policy-Guided Hybrid Simulation (PGHS), a dual-process framework combining LLM reasoning with learned behavioral policies to simulate merchant-level user behavior for counterfactual strategy evaluation without costly online experiments.
AdaSplash-2 accelerates differentiable sparse attention for transformers by using histogram-based initialization to compute the normalizer in 1–2 iterations instead of many, reducing computational overhead while maintaining input-dependent sparsity for long-context training.
Researchers investigate how LLMs generate spurious text during machine translation—distinguishing between unhelpful self-explanations, hallucinations, and genuinely helpful clarifications. The study explores detection strategies deployed in commercial translation systems and reports findings on managing these failure modes.
Researchers propose K-Token Merging, a compression technique that groups token embeddings in latent space to reduce computational overhead in LLM inference. The method uses a lightweight encoder to merge K consecutive tokens into single embeddings, then processes the compressed sequence through a LoRA-adapted model while preserving original vocabulary output.
Researchers introduced QuantCode-Bench, a 400-task benchmark for evaluating LLMs on generating executable algorithmic trading strategies for the Backtrader framework. The benchmark tests whether models can combine financial domain knowledge, API mastery, and correct syntax to produce strategies that execute on historical data.
Researchers propose IG-Search, a reinforcement learning framework that rewards LLMs for effective search queries using step-level information gain signals rather than trajectory-level rewards. The approach measures how retrieved documents improve model confidence in correct answers, addressing gradient collapse in existing search-augmented reasoning systems.
DiscoTrace, a new framework, maps how humans and LLMs construct answers to information-seeking questions using discourse acts and rhetorical structure. Analysis of nine human communities shows diverse answering strategies, while LLMs lack rhetorical variety and systematically favor breadth over human-like selectivity.
Researchers evaluated a retrieval-grounded LLM conversational agent against clinician-authored responses for CGM diabetes counseling across 12 cases, with 6 senior UK diabetes clinicians rating both approaches in a blinded comparative study conducted Oct 2025–Feb 2026.
Researchers introduce Interrogative Uncertainty Quantification (IUQ), a framework for measuring confidence in long-form LLM outputs by combining cross-sample consistency checks with within-sample faithfulness metrics, addressing a gap in uncertainty estimation for free-form text generation.
Researchers tested two approaches for encoding reusable experience in AI systems across 4,590 code-solving trials. A compact "Gene" representation outperformed documentation-heavy "Skill" packages, proving more robust to structural changes and effective as a substrate for test-time evolution.
OpenMobile, an open-source framework, enables scalable synthesis of mobile agent tasks and trajectories using vision-language models, achieving near 70% success on AndroidWorld benchmarks through environment memory exploration and policy-switching between learner and expert models.
Researchers introduced ProVoice-Bench, a new evaluation framework for proactive voice agents with 1,182 test samples across four novel tasks. Testing state-of-the-art multimodal LLMs revealed significant performance gaps, particularly in over-triggering and reasoning, exposing limitations in current models' ability to anticipate and intervene proactively.
Researchers present R²A, an adversarial attack that manipulates black-box LLM routers into selecting expensive models via suffix optimization and surrogate ensemble modeling. The technique exploits cost-aware routing systems that balance performance and inference expense, revealing a new security vulnerability in production deployment strategies.
Researchers replicated findings on how small transformers (Gemma 2B, Llama 3.2 1B) make early, irreversible commitments to decisions. Using mechanistic analysis, they identified specific attention heads that sustain these commitments across layers and found planning requires ≤16 layers but commitment needs deeper architecture.
Researchers introduce ConfGuide, a hybrid decision-making framework that uses conformal risk control to generate concise AI guidance for human decision-makers. The approach narrows outcome suggestions to reduce cognitive overload while keeping humans in control of final choices.
Researchers present a hybrid system combining LLMs with custom vocabularies to detect and explain hate speech across English, French, and Greek, prioritizing transparency and context over simple removal.
RaTA-Tool introduces a retrieval-based framework enabling multimodal large language models to select and invoke external tools from open-world settings, moving beyond text-only, closed-world tool-use approaches that struggle with unseen APIs and diverse input modalities.
Researchers introduce Text2Arch, a new dataset enabling language models to generate scientific architecture diagrams from natural language descriptions via intermediate code generation. The work addresses a gap in open-access resources for automating visual system design documentation across enterprise, software engineering, and educational contexts.
Researchers introduce XQ-MEval, a benchmark dataset spanning nine language pairs to expose cross-lingual scoring bias in machine translation metrics. The dataset uses semi-automatic error injection and native speaker validation to ensure parallel-quality translations, addressing a gap in systematic evaluation of multilingual systems.
Researchers propose IE-as-Cache, a framework that repurposes information extraction as a reusable cognitive cache to improve multi-step agentic reasoning across LLMs. The approach dynamically maintains compact intermediate information and filters noise, showing significant improvements on challenging benchmarks.
Researchers propose LongAct, a reinforcement learning technique that leverages high-magnitude activation patterns in query and key vectors to improve long-context reasoning in LLMs. The method treats long-context RL as a sparse optimization problem, drawing parallels to model quantization to identify which weights matter most for training efficiency.
Researchers benchmarked six multilingual embedding models (Potion, Gemma, BGE, Snow, Jina, E5) for hate speech detection across Lithuanian, Russian, and English using a new Lithuanian corpus (LtHate) and existing datasets, comparing anomaly detection and classification approaches.
Researchers introduce DynAfford, a benchmark for testing embodied AI agents in environments where object affordances shift dynamically and aren't explicitly stated. The accompanying ADAPT module helps planners infer hidden preconditions and adjust actions in real-world scenarios where naive instruction-following fails.