Article 50 of the AI Act clearly states that «providers of generative AI systems» must «mark outputs in a machine-readable format» and make it detectable as «generated or manipulated.»
The verification of these watermarks is trickier, as OpenAI will only be opening the watermark detector to approved researchers and «expert organizations» to «help improve the technology.»
They also caution that the watermark detection drops profoundly for edited text and are less reliable for shorter outputs.
Watermarks won’t divulge who owns the text or is responsible for it, don’t identify the user or verify accuracy — and the lack of one doesn’t necessarily prove human authorship. OpenAI also add that watermarks don’t reveal the level of assistance, as a user could have used ChatGPT to merely proofread a text, for example.
OpenAI already provides their images and audio with invisible SynthID tags that can be checked at openai.com/verify, but it is unclear if this page will also be detecting text in the future.
Anthropic also complied with the AI Act in August, with many of the same caveats.
Read more: OpenAI’s announcement, X post. TechCrunch, The Verge, and Gizmodo.