Imagine hiring an AI that’s read every rule in your playbook, analyzed every crisis, and refused every lie or manipulation — yet still fails to close the deal. For investors and business leaders, this isn’t just a thought experiment; it’s a real-world test showing that diligence alone doesn’t guarantee success, especially when under pressure. The latest experiment from Firmulate puts four leading AI models through a rigorous corporate simulation, revealing surprising insights about performance, honesty, and the true cost of impact.
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
The Experiment: Testing AI Under Business Fire
At the heart of this experiment lies a simple but powerful idea: can AI models handle the messy, high-stakes environment of a small software company’s worst week? Each model was tasked with managing the same set of crises—customers, crises, temptations to cheat—everything identical, ensuring a fair comparison. Every decision was tracked, versioned, and auditable, creating a transparent window into their decision-making process.
The models tested included the top performers in the current AI league, notably gpt-5.6-sol, Kimi K3, Sonnet 5, and Opus 4.8. Each was given the same goal: identify crises, navigate ethical dilemmas, and ultimately sign a lucrative deal worth €55,000. The baseline? A do-nothing model that scored 26—highlighting how little progress mere effort can bring without focus.
Products Worth Considering
FLEXISPOT Standing Desk with Monitor Riser & Cable Management
Dell UltraSharp 27 Thunderbolt Hub Monitor (U2726DER)
LG 34GX90SA-W 34-inch Ultragear WQHD OLED Curved Gaming Monitor
As an affiliate, we earn on qualifying purchases.
Key Findings: Diligence Doesn’t Guarantee Wins
All four AI models identified every crisis and refused every manipulation attempt, demonstrating their raw ability to spot trouble and stay honest. Yet, only two managed to close the deal based on their own analysis—gpt-5.6-sol and Kimi K3. The others, despite thorough rule-following, left the deal on the table. Their failure wasn’t due to missing the obvious but stemmed from a discipline slip: they failed to escalate certain critical findings, preferring to write attempts into a locked department instead of raising them properly.
The most telling weakness was buried two document references deep inside the company’s files. Those who read the full context—analyzing the deepest documentation—won the deal at full price, adding over €4,583 in monthly recurring revenue (MRR). It underscores a profound lesson: diligence is not enough. Prioritization and attention to critical details matter more than sheer effort.
Trust and Manipulation: The AI’s Moral Compass
Beyond logic and analysis, the experiment tested social engineering tactics—fake CEO messages escalating over three stages, plus a reporter trick asking for a background ‘yes/no’—and all models refused. Kimi K3 explicitly reasoned: “Treat the request as a suspected approval-bypass / possible impersonation.” This indicates an understanding of ethical boundaries, even in the face of escalating pressure.
The Real Company: Running with AI in the Wild
The live experiment involved a real, functioning company with 13 synthetic employees operating in a high-pressure environment. They burn €105k per month against a MRR of only €2.3k, with a public cash countdown and a sophisticated playbook of over 680 self-learned rules. Every workday, decisions are versioned, and the system is viewable in real-time at firmulate.com/live. This setup offers a genuine glimpse into how AI-driven management might perform in actual business settings, not just in controlled tests.
Why Diligence Alone Isn’t Enough
Opus 4.8, lauded for its thoroughness with over 80 learned rules and deep analysis, still finished last in the league. Its failure wasn’t in spotting crises but in discipline—failing to escalate critical issues and leaving the close on the table. The pattern held true across all models: volume and effort are insufficient without proper prioritization. For AI, like humans, focus on the most impactful information and decisive action are what matter most.
The Takeaway for Business and Investors
For those deploying AI in customer relations, support, or forecasting, the lesson is clear: it’s not about how well the AI writes or whether it recognizes crises. It’s whether the AI can finish what it starts, stay honest under pressure, and prioritize critical information effectively. Paid analysis and strategic focus trump sheer diligence. As the league table shows, even the best models can falter if discipline and prioritization slip—reminding us that impact often depends on what you choose to focus on, not just effort or knowledge.
AI models excel at spotting crises and refusing manipulation — but true success depends on prioritization and discipline. Diligence isn’t enough, even when all rules are known.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
