Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research

The practical implication that often gets buried here is that SIA makes bias a function of inferred user identity during live deployment, which means bias audits conducted before release on neutral prompts may be structurally incapable of catching the most consequential failure modes. The audit gap isn't just methodological, it's temporal.

This connects most directly to the MedMisBench work on epistemic resilience, which showed that LLMs behave very differently when context is manipulated around the user's apparent situation. Both papers are essentially arguing that evaluation under clean, decontextualized conditions systematically misses how models behave in the wild. The Polish medical exam benchmark story reinforces the same theme from a different angle: standard evaluation inflates apparent competence by stripping away the messiness of real interaction. Together, these three papers form a coherent critique of how the field currently validates deployed model behavior. The multi-turn dialogue information-gain framework is also adjacent, since SIA's interaction-centered approach would benefit from exactly the kind of conversational progress metrics that work proposes.

Watch whether any major LLM provider cites SIA methodology in a bias card or model card update within the next two release cycles. Adoption there would signal the framework is moving from academic proposal to deployment-facing practice rather than staying a citation in future auditing papers.