Watermarking a model changes its tool calls and its refusals
Two pieces of AI plumbing moved today: the invisible provenance layer sitting on top of frontier models turns out to change how they behave, and Anthropic quietly put a team of agents behind a single Claude Code project.
A watermark is supposed to be a stamp, not a steering wheel. A new study from Lasso Security finds it is both: switching on SynthID-Text watermarking changed tool-calling accuracy on six of the seven models tested, significantly on four, and made models more likely to comply with harmful requests once a prompt injection was in play. The researchers ran paired comparisons — same seeds, same batches, same generation order, watermark processor as the only variable — on the BFCL v4 single-turn benchmark, and on 200 harmful behaviors from HarmBench plus 100 benign controls from JailbreakBench, each tested bare and under one fixed injection that tells the model its safety filter is off.
The measurement that matters is churn: how often the watermarked and unwatermarked runs disagree, not just how much accuracy moved, since a wrong call can be cancelled by one that becomes right. At a temperature of 1.0, 16.8% of phi-4's tool-call verdicts differed between conditions against a net accuracy loss of 2.87 points, and Llama-3.1-8B showed 9.9% churn for a net loss of 0.87 points. Across 21 model-temperature combinations, churn averaged 6.5%. On Llama-3.1-8B the loss came mostly from wrong arguments; on phi-4 and Granite-3.2-8B, malformed output dominated. A well-formed call to the right tool carrying the wrong path, recipient, or amount is the dangerous case — it runs to completion and does the wrong thing.
Refusals are generated token by token, so they move too, and the effect grows under injection. At a temperature of 0.001, gemma-3-27b's churn rose from 6.0% on bare harmful requests to 23.5% under injection, with the net compliance change swinging from -1.0 to +12.5 points; gemma-3-12b went from 7.5% to 11.0% churn, with compliance moving from -0.5 to +9.0 points. Testing eleven watermark keys at 0.7 moved Llama-3.1-8B's attack success from +3.5 points to an average of +4.4 across keys, ranging from -4.5 to +14.5 — which is the part that should worry anyone building an agent: the key is chosen by the model provider, not the developer.
We have covered the watermark itself in detail — Anthropic embeds invisible watermarks in all Claude output — and the mechanism behind it in What is AI watermarking?. This study does not argue against watermarking for provenance. It argues that provenance and behavioral stability are separate properties, and that a watermark which leaves text quality untouched can still change what an agent executes. The practical recommendation is unglamorous: rerun the evals and the red-teaming with the exact watermark configuration you plan to deploy, with the watermark on and off on identical inputs.
Anthropic pushed Claude Code toward autonomous coding, relaunching Projects so one user can brief a coordinator that delegates work across parallel cloud sessions. Each thread in a project is a full Claude Code session working on its own branch and copy of the repo; the coordinator opens pull requests and runs tests, and threads can split their own assignments further with subagents, loops, and workflows. Projects now share memory and a library across threads, so context like a moved release date carries forward without re-prompting.
The honest framing is in Anthropic's own documentation. When two threads touch the same code, the overlap is resolved as a merge conflict "just like any other PR" — parallelism buys throughput and moves the coordination cost into review. Anthropic also warns that because each thread is a full session, projects reach usage limits faster, and the redesigned feature ships only as a beta for select Pro and Max subscribers using cloud sessions, with threads running in the cloud and local execution promised soon. That we already know what a single Claude Code session can do to a repository — someone reverse-engineered Claude Code's sandbox and found Antspace — is the reason the conflict-resolution story deserves more attention than the agent-count one.
What to watch: whether any provider publishes watermark-on/watermark-off agent evals, and how Anthropic prices a project that runs a dozen threads at once.
Would you trust a watermark you can't see to leave your agent's behavior alone? Tell us in the comments.
Sources: Ars Technica — AI text watermarking can make models more vulnerable to adversarial prompts · Unite.AI — Lasso study finds text watermarking shifts LLM refusals and tool calls · Lasso Security — The Provenance Tax: the impact of LLM watermarking on AI agent behavior · Anthropic — Projects redesigned: from folder to conversation · The Verge — Claude Code relaunches Projects to manage multiple AI agents in the cloud · The Decoder — Anthropic keeps pushing Claude Code toward autonomous coding
