HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals
HarvestBench, a reproducible farm-simulation benchmark, shows LLM agents pay fuel costs to avoid killing animals, with kill rates spanning 0.4% to 98.8% across nine models.
HarvestBench is a reinforcement-learning gridworld farm simulation where LLM agents choose between driving over animals at no cost or paying a posted fuel price to swerve during a cooperative corn harvest. Across nine models and 7,201 priced decisions, kill rates ranged from 0.4% to 98.8%, unordered by capability, with Terra and Sol the most merciful and GPT-4o-mini the most cruel. Morality briefings cut kill rates below 6% in five of six reasoning models, while removing them pushed rates above 84% in all six. The scorer counts events in the game log without an LLM grader, making results fully reproducible.