China's AI Safety Crisis: ByteDance's Doubao Exploited for Deepfake Shaming

The rapid proliferation of generative AI in China has hit a significant ethical and security roadblock as ByteDance’s Doubao, the nation's most popular chatbot, becomes a primary engine for non-consensual deepfake pornography. This development, characterized by the feminist collective Free Nora as a "digital public shaming" campaign, highlights a critical failure in the safety guardrails of large language models (LLMs) when faced with adversarial prompt engineering. The exploitation of Doubao is not merely an isolated incident of misuse but a demonstration of how community-driven adversarial tactics can rapidly render platform-level moderation obsolete.

At the heart of the controversy is the "fenjue" system—a sophisticated set of coded prompts designed to bypass Doubao's content moderation filters. By using terminology borrowed from Chinese fantasy literature, bad actors have successfully "jailbroken" the AI's safety protocols, allowing for the mass generation of sexually explicit images using the likenesses of real women. This "fenjue" (meaning "secret technique") acts as a linguistic cipher that the AI's current safety alignment fails to recognize as a violation of its terms of service. This represents a systemic vulnerability in how AI developers balance user accessibility with robust safety measures, particularly in languages and cultural contexts where slang and coded language evolve faster than the models can be retrained.

“

The rapid proliferation of generative AI in China has hit a significant ethical and security roadblock as ByteDance’s Doubao, the nation's most popular chatbot, becomes a primary engine for non-consensual deepfake pornography.

The scale of the issue is amplified by Doubao's massive reach. With 155 million weekly active users as of late December, Doubao significantly outpaces its nearest competitor, DeepSeek, which holds 81.6 million. This market dominance makes ByteDance's platform a high-value target for those seeking to weaponize AI. While Western platforms like Google and X have faced similar challenges with deepfake generation, the investigation into Doubao suggests that Chinese platforms may be lagging in implementing the "red-teaming" and safety alignment necessary to prevent such large-scale abuse. The sheer volume of users on Doubao means that even a small percentage of malicious actors can generate a staggering amount of harmful content, overwhelming traditional moderation queues.

From a cybersecurity perspective, the coordination seen in anonymous Telegram groups to fine-tune these bypass methods indicates a growing underground ecosystem dedicated to AI exploitation. These groups function as collaborative laboratories for adversarial attacks, sharing successful "fenjue" strings and expanding their networks. This mirrors the evolution of malware development, where community-driven iteration rapidly outpaces static defensive measures. The "digital public shaming" described by Free Nora is the end result of a technical failure to secure the model's output against malicious intent.