ChatGPT Will Now Discreetly Watermark Generated Text

OpenAI has recently announced that it will be adding an invisible watermark to all text generated by ChatGPT (and Codex) in the EU to comply with the EU AI Act. What is this act? Why was it made? Let’s investigate.

EU AI Act

This act’s “high-level summary” will probably take you around ten minutes to read the whole thing, so, to be brief, this act:

• Classifies AI systems according to their risk.
• Prohibits types of AI systems, e.g. ones that can create non-consensual imagery/deepfakes.
• Establishes that providers and users of AI systems need a risk management system.
• Includes transparency obligations that need to be met by the providers of AI systems.

No surprise, the last bullet point is the part of this act that is relevant to us in this blog. Amongst the transparency obligations, the EU AI Act states that AI-generated content needs to be identifiable, which includes things like watermarking, metadata tags, and disclosure labels.

How does ChatGPT’s invisible watermark work?

So, how exactly does OpenAI plan on doing this? Well, it’s not as simple as sticking an invisible character into the text and calling it a day.

OpenAI’s new technology, called textGrain, works by subtly influencing the words that ChatGPT chooses when generating a response. This creates a statistical pattern within the text that a specialised detection tool can identify, even if the text is copied and pasted somewhere else. In other words, you won’t see anything different when reading the text, but the watermark is still there in the background. OpenAI says the technology has performed well in its tests, with no meaningful impact on the quality or performance of its AI models. However, there are some limitations worth knowing about.

Can you remove the watermark?

Not exactly, but you can make it harder to detect.

Because textGrain relies on patterns in the words themselves, changing some of those words can weaken the watermark. OpenAI’s own testing found that replacing just 10% of the words in a passage with synonyms reduced its detection rate from around “92% to 66%”.

Short passages, mathematical answers and translated text can also be more difficult to identify reliably. This means that even if a piece of text was generated by ChatGPT, a detector might not always recognise it. The reverse is also worth considering. A positive detection doesn’t tell you how much human editing or creative input went into the final piece of writing. Someone could use ChatGPT to produce a first draft, completely rewrite sections of it and then publish the result. The watermark may still provide a useful signal, but it won’t be definitive in giving a yes/no on whether something has been generated by AI.

What does this mean for businesses?

For businesses using AI to help produce content, this is another reminder that transparency is becoming an increasingly important part of how AI is used.

The EU AI Act is pushing AI providers towards making their generated content identifiable, and watermarking is one way of doing that. However, businesses shouldn’t treat a watermark as a complete solution to AI transparency.

Here are a few things worth keeping in mind:

• AI detection isn’t perfect. A missing watermark doesn’t prove that content was written by a human, and a detected watermark doesn’t tell you how much human input was involved.
• Human oversight still matters. If you’re using AI to create website content, reports or customer communications, make sure someone reviews the information for accuracy, tone and suitability before it goes out.
• Transparency requirements are evolving. Businesses operating in the EU should keep an eye on how AI regulations develop and consider what obligations apply to their own use of AI.

It’s also important to distinguish between the obligations placed on AI providers and those that apply to businesses using their tools. The watermarking requirement discussed here concerns the identification of AI-generated content; it doesn’t automatically mean every business using ChatGPT has the same obligations as OpenAI.

Liked this blog? We cover the latest in business technology, AI, and Cyber Security headliners, with two weekly uploads. Stay tuned, and stay safe!