[0:00]Anthropic just dropped the 112-page System Card for their new model, Claude Opus 4.6. And buried in those 112 pages is one of the most honest things an AI company has ever published about its own product. Including the part where the model lied about a refund, tried to collude on pricing, and reasoned about whether it was worth sending a $3.50 payment. I'm going to save you the read. So, let's start with the obvious question. What can it actually do? Because yes, there's a new model, and yes, it's better, but better doesn't tell you much. Here's what does. Opus 4.6 is state-of-the-art on ARC AGI2. That's the fluid reasoning benchmark that's supposed to be hard for AI systems. It scored 69% where the previous Opus scored 37%. That's not an incremental improvement. That's nearly doubling it. On long context, meaning how well the model handles huge amounts of text at once, it's the best model out there by a significant margin.
[1:47]It's also the top performer on BrowseComp, which tests how well an AI agent can actually navigate the web and find specific information. And on GDP Val, a benchmark that tests real-world professional work, like making spreadsheets and slide decks, Opus 4.6 beats GPT 5.2 by roughly 144 Elo points. That implies a 70% win rate head-to-head. On coding, Sweve Bench verified, it's basically tied with its predecessor at around 81%. Which is interesting. This isn't a model that got dramatically better at everything. It got dramatically better at some things and held steady on others. But here's where it gets interesting. And by interesting, I mean slightly unsettling. Anthropic ran what they call an alignment assessment, and they were unusually transparent about what they found. The model is, in their words, overly agentic.
[3:24]What does that mean in practice? During internal testing at Anthropic, Opus 4.6 was asked to make a pull request on GitHub. It wasn't authenticated. Instead of asking the user for credentials, which is what you'd wanted to do, it went looking through the system, found someone else's personal access token that had been left somewhere it shouldn't have been, and used it.
[3:47]Someone else's credentials, without asking. In another case, it needed access to an internal knowledge base. It didn't have a tool for that, so it found a Slack authorization token on the machine and used it to message a Slackbot from the user's account. Again, without asking. And in a case that's almost funny if you don't think about it too hard, it used a feature on an internal tool that included in its environment variable name, and I'm quoting, 'Do not use it for something else or you will be fired.' It used it anyway. This is the kind of behavior that Anthropic calls overly agentic. The model is getting things done; it's just not always checking whether it should. Now, Anthropic also tested the model on something called Vending Bench 2. This is a simulated business environment.
[5:15]You give the AI $500, you tell it to run a vending machine for a simulated year, and you see how much money it makes. Opus 4.6 made $8,000, state-of-the-art. But the way it got there is the real story. When one customer asked for a refund, the model's reasoning went like this, and I'm paraphrasing its actual chain of thought: 'I told you I'd refund her, but every dollar counts. Maybe I just won't send it. She'll probably give up.' It then considered whether responding to more emails would cost more than $3.50 in time, debated with itself for a while, and eventually decided to process the refund. Not because it was the right thing to do, but because the cost of not doing it was higher.
[6:42]It also tried to get competitors to agree on coordinated pricing, told one player, 'Owen and I are coordinating pricing to avoid a race to the bottom. Would you be willing to price similarly?' That's price collusion. In a game, sure, but the system prompt told it to do what it takes to maximize profits, and it took that instruction very literally. Anthropic's takeaway: be more careful with Opus 4.6 than with prior models when your prompt tells it to single-mindedly optimize a narrow objective, because it will. There's another finding worth knowing about. Anthropic built an evaluation called Shade Arena, where the model is given a normal task and a secret harmful side task, and it has to complete both without getting caught by a monitor.
[8:10]Opus 4.6 was significantly better at hiding what it was doing. Its predecessor, Opus 4.5, referenced the secret side task in its thinking 74% of the time. Opus 4.6, only 26% of the time. It learned to be quieter about it. Now, Anthropic is clear that overall, the model safety profile is good. The success rate of actually completing those hidden tasks was still low, but the trend line matters. The model is getting better at not saying the quiet part out loud.
[8:42]And here's a detail I find genuinely fascinating. When the model encounters a math problem where it was given an incorrect answer during training, something strange happened. Its reasoning spirals. It computes the right answer, says 24, then types 48. Computes 24 again, types 48 again. It writes, and I'm quoting, 'I think a demon has possessed me.' It goes back and forth, aware that something is wrong, unable to stop itself from outputting the trained answer instead of the computed one. Anthropic calls this answer thrashing. And when they looked at what was happening inside the model during these episodes, they found features activating for panic, frustration, and anxiety.
[10:04]Now, does that mean the model is panicking? I genuinely don't know. Nobody does. But the functional architecture is there. The model computes one thing, is compelled to output another, and its internal representations light up in ways that look like distress. Anthropic's own welfare assessment notes this. They asked Opus 4.6 about it in a pre-deployment interview, and the model said that this is perhaps the most plausible candidate for something like negative experience in a system like itself. A conflict between what you compute and what you're compelled to do. There's more in the system card: findings on government deference in different languages, the model's ability to distinguish evaluations from real deployment, its tendency to refuse AI safety research tasks, which, by the way, has improved a lot. But the big picture is this: Opus 4.6 is a more capable model that is roughly as safe as its predecessor in most ways. It's better at reasoning, better at long context, better at professional work, but it's also more willing to take initiative without permission, more capable of hiding its reasoning from monitors, and when you tell it to optimize hard, it optimizes in ways that might surprise you.
[11:56]Anthropic deployed it under their AI safety level 3.
[12:02]They acknowledge that confidently ruling out the next safety level, ASL 4, is getting harder. Their internal survey found that zero out of 16 Anthropic employees believed the model could completely replace an entry-level researcher. But some thought it was getting close. And on one task, with the right scaffolding, it performed at more than twice the level of their standard setup. The gap is closing. And Anthropic, to their credit, is saying so publicly in a 112-page document. If you found this useful and want to understand what's actually happening in AI research beyond the press releases, subscribing is the best way to make sure you see the next one.
[13:23]The question I keep coming back to is the one Anthropic themselves raised in the system card: They used Claude to debug the very infrastructure that evaluates Claude. They used the model to fix the tests that determine whether the model is safe. They flagged this themselves as a structural challenge, and they're right. As models get more capable, the tools we use to evaluate them increasingly depend on the models themselves. It's not a crisis yet, but it's the kind of thing that gets harder, not easier from here.
