LLMSniffer: Detecting LLM-Generated Code via GraphCodeBERT and Supervised Contrastive Learning

The 16-point accuracy gap between the two benchmarks is the detail worth sitting with: GPTSniffer and Whodunit test different distributions of code, so a model that scores 94.65% on one and 78% on the other is telling you something about how fragile detection is when the training distribution shifts. The paper's real contribution may be less about the headline numbers and more about whether contrastive learning produces representations that generalize across those gaps.

This sits in a growing cluster of work on AI output reliability and attribution. The 'Diagnosing LLM Judge Reliability' paper from April 16 raised a structurally similar concern: aggregate metrics can look strong while per-instance behavior is inconsistent. LLMSniffer faces the same problem in reverse, trying to make confident per-instance calls about authorship from aggregate training signal. Meanwhile, OpenAI's Codex update covered the same week shows the supply side accelerating, which means detection benchmarks built on today's model outputs will need continuous refreshing as generation styles shift.

Watch whether LLMSniffer's accuracy holds when tested against code produced by models released after its training cutoff, specifically GPT-4o or Claude 3.5 Sonnet outputs. If performance drops significantly on post-cutoff samples, the framework is tracking stylistic artifacts of specific model versions rather than any durable signal of AI authorship.