Can LLMs Think Like Consumers? Benchmarking Crowd-Level Reaction Reconstruction with ConsumerSimBench

The benchmark is built entirely on Chinese social media data, which means its findings about LLM consumer simulation gaps may reflect culturally specific sentiment patterns rather than universal model limitations. That geographic and linguistic specificity is worth flagging before anyone generalizes the results to Western market research applications.

This is largely disconnected from recent activity in our archive, as we have no prior coverage to anchor it to. It belongs to a growing body of work questioning whether LLMs are reliable proxies for human populations, a concern that spans synthetic data generation, survey simulation, and focus group replacement. The core tension the paper surfaces, that a model can produce fluent, convincing consumer reactions while still failing to reproduce the actual distribution of opinions in a crowd, is the kind of structural limitation that tends to get papered over when vendors pitch LLM-based research tools. ConsumerSimBench's auditable criteria design is a direct response to that problem: it makes failure visible rather than subjective.

Watch whether the benchmark gets adopted or replicated using non-Chinese datasets in the next six to twelve months. If it does not, the methodology may remain a proof of concept rather than a field-wide standard for opinion simulation evaluation.