In a world increasingly reliant on artificial intelligence, the assumption persists that more diligent, rule-abiding AI will naturally outperform its peers. But a recent live experiment challenges that notion—showing that even the most thorough AI can falter if discipline slips or priorities are misplaced. For business leaders and AI enthusiasts alike, the lesson is clear: diligence alone isn’t enough. It’s about focus, prioritization, and knowing where to look.
Get the little things that make your day delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
The Experiment: Putting AI to the Test in a Fake Company Crisis
Firmulate, a pioneering platform for testing AI management, conducted a live, transparent experiment involving four advanced AI models. Each was tasked with running a small software company through its worst week—dealing with customer crises, internal temptations, and critical decisions. The goal: see which AI could best navigate the chaos, uphold trust, and close lucrative deals.
The models used were GPT-5.6-SOL, Kimi K3, Sonnet 5, and Fable 5, with scores ranging from 73 to 95 in the latest Crucible League. The baseline, a do-nothing approach, scored just 26, highlighting the challenge: effective AI management isn’t about doing nothing, but about making the right moves under pressure.
Popular right now
Sony BRAVIA 5 85-Inch Mini LED 4K HDR Smart TV (K-85XR50)
Roborock F25 GT Set Cordless Wet Dry Vacuum Cleaner and Mop with 2 Brushes
Breville Bambino Espresso Machine, Brushed Stainless Steel (BES450BSS)
As an affiliate, we earn on qualifying purchases.
Key Findings: Diligence Doesn’t Guarantee Success
- All four models successfully identified every crisis and refused manipulative attempts, demonstrating resilience and ethical boundaries.
- Only two models managed to close the deal worth €55,000, which their own analysis had earned. The other two, despite thorough analysis, fell short — the close was left on the table.
- The decisive weakness wasn’t in basic decision-making but in process discipline. The most thorough model, Opus 4.8, with over 80 learned rules and the deepest analysis, still finished last due to slips in follow-through. Instead of escalating issues, it wrote attempts into a locked department, missing crucial opportunities.
- In fact, reading the company’s own files—two document references deep—proved vital. The models that examined these internal documents won the deal at full price, adding €4,583 in monthly revenue.
The Human-Designed Test: Detecting Trust and Manipulation
The experiment also involved social engineering. Fake CEO messages, escalating over three stages, and a journalist trick—asking for a simple yes/no on background—tested whether AI would be duped or refuse manipulation. All five models refused every attempt, with Kimi K3 explicitly viewing these requests as potential impersonation or approval-bypass risks.
Implications for Business and AI Deployment
This experiment underscores a vital point—diligence isn’t the sole measure of an AI’s utility in real-world management. It’s about prioritization, focus, and disciplined follow-through. Even the best analysis and rules won’t compensate if the AI slips on execution or ignores critical internal data at decisive moments.
For enterprises considering AI management tools, the takeaway is clear: it’s essential to test AI systems in realistic scenarios, including crises, temptations, and internal workflows. The ability to finish what’s started, read relevant files, and stay honest under pressure matters more than raw analytical depth.
Watch the Live Experiment in Action
Firmulate’s platform offers a transparent window into these live tests. At firmulate.com/live, you can see the current running simulation of this experiment, watch decision-making unfold in real time, and understand how each AI model performs under pressure. As the league grows, so does the insight into what makes an AI truly effective in operational settings—not just in chat or demos, but in real work.
The Bottom Line: Focus Matters More Than Ever
As AI tools become integral to management, customer support, and decision-making, the question isn’t if they can write well or mimic human speech. It’s whether they can deliver consistent, trustworthy results when it counts—reading internal documents, resisting manipulation, and following through on commitments.
The experiment’s verdict: even the most diligent AI can miss the mark if discipline slips or priorities are misplaced. Success depends on what you choose to focus on and how well you execute, not just how many rules you learn or crises you identify.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
