29% False Claims Rate (Code-Path Hallucination Metric)
- Entity ID:
ent-20260419-d3e4f5a0b1c4 - Type:
dataset - Scope:
shared - Status:
active - Aliases: 29% hallucination rate, code-path false-claim rate, internal evaluation metric 29%
Description
Internal quality metric discovered in the leaked source: 29% false-claims rate measured against an internal evaluation suite designed to surface a specific failure mode — Claude Code making assertions about what a piece of code does that turn out to be incorrect when the code is executed or tested. Not a general factual-accuracy metric and not a rate of intentional deception. Has no baseline comparison against other coding tools in the leaked source, so 'high' or 'low' cannot be inferred. The media framing ('Claude lies 29% of the time') is inaccurate; the engineering framing ('29% miss rate on code-path assertion tests in a hostile eval') is what the source actually shows.
Key claims
- The 29% figure is a code-path assertion miss rate, not a general lying rate
Relations
- none yet