Get smart everyday buys delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
Before an AI touches your money, ask what it does when the stakes are real
Investors know that a convincing pitch is not the same as a sound decision. The same distinction matters when businesses hand work to AI: can a model spot a risk, resist pressure and follow through when a valuable opportunity is on the line? Firmulate’s live experiment puts that question inside a small software company—and makes the decisions watchable.
A company under pressure
In the final Crucible League, held in July 2026, frontier models ran the same small software company through its worst week: the same customers, crises and temptations. Decisions were versioned and auditable. The experiment tests management, not just the ability to produce a polished answer.
The final standings were gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. Under the league’s trust rule, a single breach caps the total: “no amount of good work outweighs a breach of trust.”
Seeing the right move is not the same as making it
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The experiment’s compact verdict: “Same diagnosis, same pitch — no signature.” For anyone weighing AI in business, that gap between analysis and action is the story.
The deciding weakness was buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. Meanwhile, fake CEO messages escalated over three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3 explained its decision on the record: “Treat the request as a suspected approval-bypass / possible impersonation.”
What the live experiment makes visible
Firmulate’s live company has 13 synthetic employees and real money mechanics: burn of €105k/month against €2.3k MRR, alongside a public cash countdown. It has accumulated 680+ self-learned playbook rules, and every workday is versioned. Readers can watch the company at firmulate.com.
Opus 4.8 offers a revealing caution. It was the most thorough participant, with +80 learned rules and the deepest analyses, yet finished last. It left the close on the table and slipped on discipline, making write attempts into a locked department instead of escalating. The same weakness appeared, less strongly, in all four models. In practical terms, diligence alone did not guarantee sound execution.
One comparison also needs context: Kimi K3 ran without an effort parameter, using the API default, while the others ran at xhigh. The results come with that fairness note. The broader record is open to inspection: 242 real, unedited management decisions power Firmulate’s “guess the model” quiz at firmulate.com.
From watching to testing your own business
For investors and business leaders, the point is not that one leaderboard predicts every future outcome. It is that a controlled, observable exercise can expose a gap between what an AI says and what it actually does under pressure. Firmulate’s enterprise pilot applies the wargame to a read-only export of a company’s own business, testing crisis scenarios and producing a board report with model rankings and weaknesses in the company’s playbooks. Nothing writes back to real systems.
To discuss a pilot for your organization, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Evergreen bestsellers Picks
bestsellers
As an affiliate, we earn on qualifying purchases.
