The EU AI Act made watermarking a compliance question, but real content rarely stays a single generation event. It moves through models, rewrites, and edits before publication. This piece uses code as a stress test to argue the better question is who contributed, what changed, and who's responsible.

Watermarking can tell us something about if AI generated an output. But what happens after that output starts moving through models, humans, standards and systems?

The European Union's Artificial Intelligence Act (EU AI Act) has turned AI watermarking into more than a research topic. Since August 2, 2026, Article 50 has required providers of systems that generate synthetic text, images, audio, or video to make their outputs machine-readable and detectable as artificially generated or manipulated, subject to technical feasibility and the limits of the medium. The Act doesn't prescribe a single algorithm, though. It leaves the door open to a range of techniques: watermarks, metadata, cryptographic provenance, logging, and fingerprints.

That got me curious about what "watermarking text" actually means, and the deeper I went, the more interesting the problem got.

The watermark is in the math

One of the foundational approaches comes from John Kirchenbauer and colleagues in their 2023 paper, A Watermark for Large Language Models. The idea is great.

A large language model (LLM) generates text one token at a time. Before it picks the next token, their approach pseudo-randomly splits the possible tokens into a green list and a red list based on the preceding context. Green tokens then get a small boost to their logits, the scores the model uses before it turns them into probabilities.

Nothing is hidden inside the text. No invisible Unicode characters, no metadata. The watermark emerges from the model making slightly more green-list choices than you'd normally expect. Over enough tokens, a detector that knows the rule can reconstruct the lists and test whether the share of green text is statistically higher than chance, without ever touching the underlying model.

The clever part is that the watermark doesn't fight the model when there's only one obvious answer. With creative prose, where entropy is high, there may be many equally good next tokens, so the watermark has room to nudge the choice. With an idiom, a fact, or a tightly constrained phrase, where entropy is low, one token can dominate so strongly that the watermark effectively steps aside.

Technically, I find this brilliant. But it made me wonder what happens when AI-generated content stops being a single generation event.

The watermark meets the real workflow

Consider how content actually gets made today:

Model A generates watermarked text. Then a completely different model gets a routine instruction:

"Rewrite this in our editorial style without changing the meaning."

No one is attacking the watermark. The second model may not even know it exists; it's just generating another sequence of tokens.

This is where it gets interesting, because rewriting doesn't necessarily destroy a statistical watermark. Follow-up work by Kirchenbauer and colleagues found the watermark could survive both human and machine paraphrasing, but that paraphrasing diluted the signal, so confident detection sometimes needed considerably more text.

That distinction matters. The problem isn't that watermarking fails; it's that the artifact now has a history. Translation, summarization, personalization, corporate style enforcement, human editing, another model in the loop: any of these can transform it, and all of them are legitimate. At some point, asking only whether something was watermarked tells you less than you actually want to know.

Code makes the paradox easier to see

Code is a useful stress test because entropy disappears very quickly.

Say an enterprise application already defines:

An AI can't casually decide this is equally good:

It reads fine, and in isolation it might even be cleaner. But only the first satisfies the existing application programming interface (API) contract.

Now add framework rules:

Or an organizational convention:

There might be several plausible names:

But the organization may allow only one. And then there are cases with almost no freedom at all:

The model can't creatively replace String, getTitle(), return, or an interface-defined method just because some alternative happens to strengthen a statistical signal.

And generation is only the beginning. Code then passes through formatters, lint fixes, developer edits, static analysis, security remediation, refactoring, and pull-request review. A block like:

may legitimately become:

or, following a house convention:

Same intent, different token history. Nobody is trying to defeat AI detection. They're just engineering software.

(I'm using code here as a technical stress test, not as a claim that every source-code generation scenario necessarily falls within Article 50.)

Publishing has the same problem

Now swap the code for an article:

Journalist idea → AI research → AI draft → human rewrite → fact check → editor → content management system (CMS) → publication

Is the final article AI-generated? What if AI produced 70% of the first draft but only 15% of those words survived? What if the journalist wrote every sentence but AI restructured the argument? What if one model produced the draft and another rewrote it?

The AI Act itself recognizes some of this ambiguity. For text published to inform the public on matters of public interest, Article 50 sets a disclosure obligation but carves out an exception when the content has undergone human review or editorial control and a person or organization holds editorial responsibility for publication.

Responsibility matters, and that points toward a bigger architectural question.

Detection looks backward

Provenance moves forward. A detector receives the final artifact and tries to reconstruct its origin. Provenance does something fundamentally different.

Detection tries to reconstruct the past; provenance records the journey as it happens. That journey could preserve things like:

• which model generated something,

• which model transformed it next,

• what a human changed,

• which sources were verified,

• who reviewed it,

• and who ultimately approved it and took responsibility for the artifact.

This is also why I think the EU's broader framing is important. It explicitly talks about combinations of watermarking, metadata, cryptographic provenance, logging, and fingerprints rather than betting everything on one indestructible watermark. The Act even drops the provider-side marking obligation where an AI system is only doing standard editing or doesn't substantially alter the user's input or its meaning. That is much closer to how AI actually gets used.

From AI watermarking to AI lineage

Watermarking is a powerful idea. It creates a signal at the moment of generation. But AI is moving well beyond isolated generation, and enterprise workflows increasingly look like this:

Human → Agent → Model A → Model B → Tool → Human → Enterprise System → Final Artifact

So eventually, asking "which model wrote this?" may be the wrong question. The more useful ones are probably who contributed, what changed, what was verified, who approved it, and who ultimately takes responsibility.

That is why I think the next trust layer for generative AI will have to go beyond watermarking.

As AI moves from single generations into chains of transformation, that distinction may matter far more than the watermark itself.

FAQ

Does a statistical watermark survive translation or paraphrasing?

Mostly, yes. Follow-up research from Kirchenbauer and colleagues found the signal survives both human and machine paraphrasing, but it thins out. Detecting it confidently after heavy rewriting takes a lot more text than a fresh, unedited sample does.

Does the EU AI Act require every AI output to carry a watermark?

No. Article 50 says AI-generated outputs have to be machine-readable and detectable as synthetic, as far as that's technically feasible, but it doesn't pick one method. Providers can use watermarks, metadata, cryptographic provenance, logging, or fingerprints, on their own or combined.

Is AI-assisted code subject to the same disclosure rules as AI-generated text or images?

The article treats code as a technical stress test, not a settled legal question, because it's still unclear whether a given code-generation scenario falls under Article 50. The point is really about architecture: code has so little token-level freedom (APIs, framework rules, naming conventions) that watermark preference rarely survives contact with the need to be correct.

Why does human editorial review exempt some AI-assisted content from disclosure?

Article 50 carves out an exception for content that's gone through human review or editorial control, where a person or organization takes editorial responsibility for publishing it. The logic is that what the rule cares about is responsibility, not which tool produced the words.

What's the difference between AI detection and AI provenance?

Detection works backward: it takes a finished artifact and tries to figure out whether AI was involved. Provenance works forward: it records the journey as it happens, which model generated something, what a human changed, who verified it, and who finally approved it and took responsibility.

References

Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I. & Goldstein, T. A Watermark for Large Language Models, ICML 2023.

Kirchenbauer et al. On the Reliability of Watermarks for Large Language Models, ICLR 2024.

European Union Regulation (EU) 2024/1689, Artificial Intelligence Act, Article 50 and Recitals 133-135.

European Commission Code of Practice on Transparency of AI-generated Content, 2026.