LGMT tests whether LLMs contradict equivalent logical queries
Across 3,091 metamorphic groups, six widely used LLMs showed inconsistency rates from 20.84% to 29.44%.
Stories per day · 30 days
2 stories, newest first · one card per story
Stories tagged #logical-reasoning
Competence-based eval finds GPT and Qwen fail ordering tasks
Even strong models fail to apply the same ordering competence across presentations, despite good accuracy in bounded ranges.