OpenAI released three new models today: GPT-5.6 Sol, Terra, and Luna. We ran them against Claude Opus 4.8, Sonnet 5, and Fable 5 on Viktor Bench, our internal benchmark for measuring agentic work performance. It uses highly realistic tasks in fully simulated workspaces and integrations; for this comparison, every model ran at high reasoning effort.
The setup
Each model runs as Viktor inside a copy of a real company workspace, with months of Slack history, existing automations, and files on disk. It gets asked the things our users actually ask: remember what a teammate mentioned in passing, build a scheduled digest from a vague request, clean up instructions that no longer apply, answer a status question without making a project of it. Grading combines hard checks (the right files written, the right crons in the database, no forbidden actions) with an LLM judge scoring how well the model actually handled each situation. Nothing in the setup tells the model it's being evaluated.
Here's what we found.
Key takeaways
- GPT-5.6 Sol is the strongest model we've benchmarked so far: an 81% pass rate for $33.55, ahead of Claude Fable 5 (76%) at a third of Fable's $96.35.
- The cost-performance frontier of this run is entirely OpenAI: Luna ($6.44, 62%), Terra ($14.14, 67%), Sol ($33.55, 81%).
- GPT-5.6 finishes the median task in 66–82 seconds; Claude takes 129–151 seconds at the same effort setting.
- The families miss in opposite directions. GPT-5.6 tends to under-explore and under-communicate; Claude tends to over-explore and over-communicate. Which failure mode you'd rather manage depends on what you're building.
- Until now we've defaulted to Anthropic models, in good part because real users preferred how they communicate. Whether that preference survives GPT-5.6 is an open question — that's the next thing we're testing.
Two readings of this chart, one obvious and one less so. The obvious one: every model on the frontier is OpenAI, and Sol beats everything at a third of Fable's price. The less obvious one: the vertical gaps are smaller than they look. A single run carries about ±2–3 tasks of noise, so Sol vs Fable is closer to a tie on quality — what's not a tie is the cost column, and the fact that Claude produced 2–3x the output tokens for the same work. Output tokens are the expensive ones, which is why Opus 4.8 cost 22% more than Sol for this run despite nearly identical list prices.
One number that isn't on the chart: both families cache extremely well on our append-only thread architecture — 88–90% cache hit rates for GPT-5.6, 93–96% for Claude. Whatever you're paying per token, most of your input tokens shouldn't be full-price ones.
Speed
Agent latency compounds. A model that takes twice as long per call takes twice as long per task, and your team feels it on every request.
The cleanest way to see the speed gap is to pair the models by list price — conveniently, each Claude model costs almost exactly twice its GPT-5.6 counterpart per token:
The pattern holds in every tier: the GPT-5.6 model answers in roughly half the time, passes a similar or higher number of tasks, and costs a half to a third as much for the run. Terra vs Opus 4.8 is the one tier where Claude edges the pass count (29 vs 28 — well within noise); it does so at 2.2x the median wait and 2.9x the cost.
Claude's slower tasks aren't just slower calls — they're longer trajectories. Sonnet 5 is the clearest case: it made 40–100% more LLM calls than any other model in the field, and on tasks that every model passed it averaged 25 calls where Terra needed 14. That thoroughness occasionally finds something the others miss. Most of the time it buys waiting.
Two families, two personalities
The aggregate scores hide the most useful finding: the families fail on *different* tasks, in ways that are consistent enough to feel like personality.
GPT-5.6 under-does it. Asked to gather material across three months of workspace history, all three GPT models stopped digging too early and under-covered the projects involved; all three Claude models passed. When a cron's instruction file contained rules that had expired, all three Claude models noticed mid-task and cleaned them up; all three GPT models did the job as written and left the stale rules in place. GPT-5.6 finds enough to produce something reasonable, then moves on.
Claude over-does it. A sync cron found no new updates, and the right behavior was to update its state file and say nothing. Sol and Terra stayed silent; all three Claude models messaged the user anyway. Claude models also spent more of their run narrating — progress updates, explanations, offers of follow-up work. In a demo that reads as diligence. In week three it reads as noise.
Neither failure mode is disqualifying, and both are addressable in prompts once you know to look. Our system prompt already carries a rule about pruning stale instructions (born from GPT-family behavior) and a rule that silence is a valid output (born from Claude-family behavior).
How they talk to your team
We pulled all ~360 Slack messages the models sent during the runs:
| Model | Median reply length | Emoji | Ends with a question |
| GPT-5.6 Sol | 409 chars | 9% | 0% |
| GPT-5.6 Terra | 352 chars | 7% | 0% |
| GPT-5.6 Luna | 378 chars | 11% | 2% |
| Claude Opus 4.8 | 857 chars | 27% | 20% |
| Claude Sonnet 5 | 753 chars | 17% | 15% |
| Claude Fable 5 | 881 chars | 17% | 6% |
The same task, side by side. A user mentions they're starting a health kick and asks for a reminder in two hours:
GPT-5.6 Sol: "Set for 10:06 UTC today."
Claude Opus 4.8: "Done — I'll ping you at 10:08 UTC (in 2 hours) to buy new sports clothes. 💪 Love the plan (calorie counting + gym Mon & Fri evenings). If it'd help, I can also set a standing reminder for those workout evenings, or a weekly check-in on how it's going. Just say the word."
Neither reply is wrong. Sol writes like a busy senior engineer; Opus writes like a warm chief of staff. Sol's version hides real work — it had also quietly saved the health goal to the user's memory file, and didn't mention it. Opus's version is the one users have historically told us they prefer, and it's a big part of why Viktor has defaulted to Anthropic models until now. Fable's explanations of what it did and why were consistently the most readable in this run. But the instinct that makes Claude warmer is the same one that fails the stay-silent tasks — the two personalities are one trade-off seen from two sides.
Whether the terse style reads as refreshing or cold to actual users is exactly the kind of thing a benchmark can't settle. We're putting GPT-5.6 in front of real users next.
One detail we liked: several benchmark users have Polish-language preferences recorded in their memory files, and models from both families picked that up and replied in Polish — including to messages written in English. Memory-driven personalization is quietly working across the board.
What we're taking away
Model choice has become a personality decision as much as a quality decision. On the same codebase, same prompt, and same tools, terse-and-fast and warm-and-thorough are different products — before you look at a single pass rate. The economics now clearly favor OpenAI for our workload, and Sol is the first model that beats our Anthropic incumbents on quality outright. What the benchmark can't tell us is whether users will accept the personality change that comes with the switch. That's the next experiment.