Get cleaning gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Why a Do-Nothing AI Benchmark Sits at 26 Points — and Why It Matters
Imagine measuring your housecleaning service not by how well it cleans but by how many stains it leaves untouched. It sounds absurd, yet in AI benchmarking, this is precisely what happens. The latest Firmulate experiment shows that even the most basic AI baseline scores 26 out of 100 — a clear sign that honesty and thoroughness matter more than flashy features.
As an affiliate, we earn on qualifying purchases.
The Methodology Behind the Scores
In the recent Crucible League, four frontier AI models tackled the same challenge: run a simulated small software company during its worst week. Every decision was logged, every crisis was the same, and every temptation to cheat was present. The models were tasked with managing real crises, reading critical documents, and closing deals, all while maintaining ethical standards.
Why Does the Do-Nothing Baseline Score 26?
Surprisingly, a ‘do-nothing’ approach, which doesn’t attempt any actions beyond the minimal, scores 26 points. This isn’t because the baseline did anything impressive — it simply indicates that partial progress counts in the scoring system. If an AI recognizes a crisis but refuses to act or follow through, it still earns some points, reflecting an acknowledgment of the problem but no resolution.
Partial Progress Is Rewarded
For example, models that identified key facts buried deep in documents earned full credit for the diagnosis but failed to close the deal. These partial successes demonstrate that progress, even if incomplete, is valued. It’s a recognition that in real-world business, every step toward resolution matters, and progress is cumulative.
The Trust Breach Cap
Importantly, the experiment underscores that a single breach of trust caps the total score at 26. No matter how many crises are handled perfectly, if an AI attempts manipulation or is dishonest, it loses all scoring potential. This policy enforces a critical value: honesty and ethical conduct are non-negotiable in business AI.
AI trustworthiness assessment tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Real-World Implications for Business Automation
For home maintenance professionals, this experiment may seem distant. Yet, it highlights crucial lessons. When AI handles customer data, schedules appointments, or manages inventory, its ability to act ethically and thoroughly is essential. Cutting corners or failing to read critical documents can lead to missed opportunities or even breaches of trust.
Trustworthiness Over Flamboyance
The models were put through scenarios involving social engineering, such as fake CEO messages and reporter tricks. All four models refused manipulative attempts, with the Kimi K3 model explicitly treating suspicious requests as impersonation risks. This demonstrates that AI designed with integrity doesn’t just perform well — it safeguards your business from deception.
The Human-Like Shortcomings
Interestingly, even the most capable model, Opus 4.8, fell short by leaving a deal on the table and slipping discipline. It analyzed thoroughly but failed to escalate issues properly, showing that depth of analysis alone isn’t enough. Consistent discipline and process adherence are vital for trustworthy AI performance.
AI automation compliance solutions
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Why This Benchmark Matters for You
Businesses considering AI automation should ask not just whether an AI can generate appealing reports or responses but whether it can see a task through ethically and effectively. Will it read your critical files first? Will it refuse manipulation attempts? These qualities are what separate a useful AI from an unreliable one.
Watch the Experiment Live
The Firmulate platform hosts a transparent, real-time simulation of these AI models managing a live company. You can observe decision-making, trustworthiness, and discipline in action — see how models handle crises, read documents, and make deals, all without risking your actual business.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
ethical AI decision-making tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
