Tokenizer Fertility and Zero-Shot Performance of Foundation Models on Ukrainian Legal Text: A Comparative Study

The Ukrainian legal corpus is doing real work here beyond localization testing. Legal text is dense, formulaic, and terminology-heavy in ways that stress tokenizer vocabulary design, so fertility gaps that look modest on English benchmarks get amplified. This makes the 1.6x fertility spread a more credible signal than a general-domain comparison would produce.

The core argument, that parameter count is a poor proxy for value, runs directly parallel to what we covered in 'Small, Private Language Models as Teammates for Educational Assessment Design,' where smaller locally-deployed models matched larger ones on pedagogical tasks. Both papers are building the same evidentiary case from different angles: deployment context and efficiency metrics matter more than headline scale. The few-shot degradation finding also complicates the picture from 'Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidance,' which showed strong results from minimal demonstrations. That work focused on training-time use of few-shot examples, not inference-time prompting, so the tension is real but not a direct contradiction. Still, practitioners using few-shot prompting as a default should treat the 26% degradation figure as a flag worth investigating in their own pipelines.

If Qwen releases a tokenizer update that closes the fertility gap with Llama-family models within the next two release cycles, watch whether the zero-shot performance advantage Llama currently holds narrows proportionally. That would confirm tokenizer design, not model architecture, is the primary driver of the efficiency differential.