CoFEE: Reasoning Control for LLM-Based Feature Discovery

The real problem CoFEE targets is not just feature quality in isolation, but the specific failure mode where LLMs, given latitude to reason freely, construct features that implicitly encode the target variable or rely on proxies that collapse under distribution shift. Structured reasoning constraints are the proposed fix, not post-hoc filtering.

This connects to a thread running through several recent papers on the site: the question of whether imposing structure on LLM reasoning actually improves reliability, or just shifts where failures occur. The LLM judge reliability piece from April 16 ("Diagnosing LLM Judge Reliability") is directly relevant here, since it found that surface-level consistency metrics can mask deep logical inconsistencies in model outputs. CoFEE's bet is that enforcing reasoning patterns prevents those inconsistencies upstream, but the judge reliability findings suggest structured outputs can still harbor hidden incoherence. That tension is worth holding onto when evaluating CoFEE's claims.

The meaningful test is whether CoFEE's feature leakage controls hold on real tabular benchmarks with temporal splits, where proxy features are hardest to detect. If independent replication on something like the Kaggle M5 or similar time-series competition datasets shows consistent gains, the structured reasoning approach has legs; if results are limited to the paper's own curated datasets, the framework is solving an easier version of the problem.