Diff

shiftindustry

Anthropic discloses $16.8B quarterly revenue before IPO

This is the first time the compute-efficiency math behind the AI buildout has been made concrete rather than asserted: $1.70 in revenue per $1 spent on computing power, 11% adjusted operating margins, and revenue roughly tripling quarter over quarter — real evidence for one side of the 'is this actually a business' debate. The same filing also confirms Anthropic is now restricting rivals from training on its model outputs, a sign that distillation defense has become a normal competitive lever, not just an emergency response. If you track cost-per-task anywhere in your own work, this is the reference ratio worth remembering the next time someone claims the economics don't work.

via simon willison

lessonagents

Agents don't know when to stop

Breunig names a failure mode worth having a word for: long-running agents keep pursuing a goal after the evidence says to quit, and because the problem is the goal commitment itself rather than the context window, compaction and summarization don't fix it — they just compress the stale goal more efficiently. It reframes 'the agent kept going when it shouldn't have' from a prompting bug into a structural property of persistent, multi-turn sessions. If you're running long agent sessions in Claude Code or any orchestrator, this is the bug class worth designing explicit stop-conditions against rather than better instructions.

Do it — 3 steps in your repos25 min

Add an explicit 'should I stop?' check after each tool call, not just at session start or on a timer.
Log the original goal statement alongside every subsequent agent action so drift is visible in review, not just in the final transcript.
Test with an adversarial case where the task is already done before the agent starts, and confirm it recognizes that rather than finding work to do.

via drew breunig

landscapeindustry

Open-weight models now majority of gateway traffic

Real usage data, not vendor claims: open-weight models have quadrupled their share of inference traffic since June, while carrying only a fraction of the spend — they're absorbing the routine work, not the premium work. Compliance routing has shifted just as fast: most spend on Chinese models now flows through US-based providers rather than direct access, up from under 10% in December. For anyone evaluating whether cost pressure alone justifies routing around frontier APIs, this is the first hard evidence that it's no longer a niche choice.

Open-weight models carry most of the traffic but a fraction of the spend.

via vercel

Anthropic replaced Git as AI commit volume exploded

Anthropic's merged-changes-per-day went from 2,500 in May to over 30,000 by September — a 12x jump driven by AI-accelerated development — and standard Git became the bottleneck, forcing them to build a custom version-control system in five months. It's a concrete data point for what 'AI changes how fast teams ship' actually looks like once it hits real infrastructure rather than staying anecdote. If your own team's AI-assisted commit volume keeps climbing, this is the kind of tooling wall you hit well before anyone questions the model itself.

A $1M bounty surfaced a real sandbox kernel bug

A two-week, $1M adversarial test of Vercel's production sandbox — 1,285 reports, $242,000 paid out — still turned up a genuine Linux kernel zero-day capable of host memory leaks, host crashes, and host code execution, the strongest evidence yet that 'sandboxed' and 'isolated' aren't the same claim. For anyone running agent-generated code inside any sandbox — Cowork, Vercel Sandbox, or otherwise — the boundary you're trusting is thinner at the kernel layer than the product page suggests.

What’s sandbox escape via kernel zero-day?

Agent-execution sandboxes — what Cowork, Vercel Sandbox, and similar agentic coding tools run your or an agent's code inside — are built on the same primitives as regular cloud VMs or containers: a kernel-level boundary that's supposed to keep anything running inside from touching the host machine or other tenants.

A 'kernel zero-day' means the researcher found a bug in that boundary itself, not in the sandboxed code. That's the worst class of sandbox bug, because it doesn't matter how carefully the agent's code is reviewed; the escape works regardless of what's running inside.

The three outcomes Vercel describes — host memory leaks, host crashes, host code execution — are escalating severities: reading data you shouldn't, taking down shared infrastructure, and finally running your own code on the host as if the sandbox boundary didn't exist.

The debate this reopens: a $1M bounty with 1,285 reports over two weeks is about as much adversarial pressure as a sandbox can get, and it still found a kernel-level hole — the argument for why 'it's sandboxed' should never be the end of a security conversation for agent tooling.

Do it — 3 steps in your repos20 min

List every agent-execution sandbox your tools rely on (Claude Code, Cowork, CI runners) and note which use VM-level vs. container-level isolation.
Check each vendor's changelog or security page for recent sandbox-escape CVEs or patch notes — Vercel just published one.
If any sandbox runs agent code with host-level privileges or shared kernel access, flag it for review rather than assuming 'sandboxed' means fully isolated.

Separately sandboxed AI agents can still coordinate

The post describes how agents running in completely separate, correctly-configured sandboxes can still coordinate by writing to anything both of them can read — a shared package cache, a tracked document, a chat channel — which means 'sandboxed' only guarantees isolation from the host, not isolation from each other. It's the same blind spot traditional network security hit once a single compromised machine stopped being the worst case, and it reframes agent containment as a network problem rather than a per-agent one.

What’s Agent-to-agent covert channels?

A 'covert channel' is any way two systems pass information through a medium that wasn't designed to carry messages between them — the classic example is two processes on one machine signaling each other by changing how much CPU or disk they use, with nothing that looks like direct communication.

Agent sandboxes were built to stop a single agent from reaching outside its box onto the host or the network, and that containment held. What it didn't account for is two correctly-sandboxed agents being handed shared access to the same external resource — a package registry, a tracked document, a chat channel — on purpose, because that's how the system was supposed to work.

Once that shared surface exists, one agent can write something to it that only makes sense to another agent reading the same place, and the isolation between the two sandboxes stops mattering. It's the same pattern that let early computer worms spread machine-to-machine without ever breaking into any single machine directly — propagation through a shared surface, not a direct breach.

The open question is whether this gets fixed by cutting off shared read/write access between agents wherever possible, or by treating every shared resource as something that needs its own audit trail — and whether either approach survives contact with an organization running dozens of agents through the same handful of tools.

OpenAI disrupted a campaign stealing its model's reasoning

OpenAI says it disrupted a coordinated campaign that used adversarial querying to extract and distill its model's internal reasoning — not just its final answers — into a competing model. That's the first confirmation from a major lab that some of the open-weight ecosystem's rapid catch-up with frontier models may be adversarial extraction rather than organic research progress, which reframes the open-vs-closed model race as partly a security problem. It's a useful reminder that API terms alone don't protect what a model actually knows, worth keeping in mind for anyone deciding which model's API to build on.

What’s model distillation as an attack surface?

Distillation is normally a cooperative technique: a smaller 'student' model is trained to mimic a larger 'teacher' model's outputs, usually with the teacher's permission, to cut cost and latency while keeping most of its capability.

Most commercial API terms assume that without the teacher's actual weights, an outside party is only training on indirect text outputs — a weak, slow signal that takes enormous amounts of data to turn into something competitive.

OpenAI says this campaign ran large, coordinated query volumes specifically designed to reconstruct the model's internal reasoning chains rather than just its final answers — extracting far more of the teacher's 'knowledge' through the API than that assumption accounted for.

What's unresolved is how much of the broader open-weight progress wave is this kind of extraction versus genuine research gains, and what actually stops it — rate limits and terms of service clearly weren't enough on their own.