OpenAI has reportedly had a system in place for watermarking text generated by ChatGPT, along with a detection tool, for about a year, as noted by The Wall Street Journal. However, there is internal debate over whether to release it. While it seems like a responsible move, it might negatively impact the company’s revenue.
The watermarking works by subtly altering how the model predicts subsequent words and phrases, creating a detectable pattern. This process, though simplified, is akin to what Google has detailed in its Gemini’s text watermarking.
Providing a method to identify AI-written content could be beneficial for educators wanting to prevent students from submitting AI-generated assignments. According to the Journal, OpenAI found that watermarking did not degrade the quality of ChatGPT’s output. In a survey conducted by the company, a global audience supported the idea of an AI detection tool by a significant margin of four to one.
Following the Journal's article, OpenAI confirmed their work on text watermarking in a blog update observed by TechCrunch. The company claimed their technique is highly accurate (99.9% effective, based on internal documents) and resistant to tampering, like paraphrasing. Nevertheless, they acknowledged that rewording using another model could easily bypass the watermark, raising concerns about its effectiveness against malicious use. Additionally, OpenAI is wary of potentially stigmatizing the use of AI tools, especially for non-native English speakers.
Despite these concerns, nearly 30 percent of surveyed ChatGPT users indicated they would use the software less if watermarking were implemented. This sentiment has led some OpenAI employees to suggest exploring less controversial, albeit unproven, methods. In its blog update, OpenAI mentioned it is in the early stages of testing embedding metadata, a cryptographically signed method that might avoid false positives but still requires validation.
