In China, deepfake fraud has been at the centre of a number of high-profile scams. This relatively new technology – improving rapidly with advancements in artificial intelligence – enables anyone to recreate a person’s voice and moving facial features. The applications are numerous and often convincing.
Earlier this year, much-loved celebrity diver Quan Hongchan (全红婵) was seen on the internet promoting free-range-eggs produced on her family’s farm. By the time the video was exposed as a deepfake, 47,000 people had already parted with their money. In another high-profile case, an employee at a Hong Kong-based company sent HKD 200m (about US$ 25.7million) to a fraudster’s account after receiving orders from a deepfake mimicking her boss. These cases are two in many.
China’s lack of transparency around crime figures makes the extent of the problem very hard to pin down, but what’s certain is that deepfake scamming has made the country’s rapid adoption of technology both a blessing and a curse.
Where convenience invites crime
Anyone who’s spent serious time in China will tell you of Chinese people’s affection for convenience. Ask a local what’s best about living in Shanghai or Beijing and their answer likely won’t be world-class restaurants or cyberpunk aesthetics. They’ll probably say: ‘It’s so convenient.’
Much of modern Chinese life’s convenience resides on the screen of your smartphone. In a thumb’s tap you can order dinner, call a taxi home, even pay your rent. Technology has integrated with the actions of daily life far more than it has in the west. The speed at which Chinese people have taken to this way of life is admirable but makes them especially vulnerable to online scams. High volumes of online transactions and a massive digital economy only adds to the problem.
What’s to be done? The past couple of years have seen the roll out and enforcement of a series of laws designed to tackle the issue of AI deepfakes. Now all AI-generated images, audio and video must carry identifiers. These can be embedded in the image or in the genetics of the content’s metadata.
Streaming services like Bilibili (哔哩哔哩), Kuaishou (快手) and Douyin (抖音 – TikTok) have all moved to comply. Douyin even said it would add its own markers to content if creators didn’t do it themselves. However, the future of deepfake fraud prevention isn’t as simple as that.
Fraud faster than detection
Deepfake technology is moving at a staggering pace. In many cases, the legislation laid out to protect citizens from it will be obsolete a few years down the line. Algorithms capable of generating hyper-realistic images, videos and audio are improving so quickly that even experts struggle to distinguish deepfakes from reality.
Another problematic reality is that criminal networks often operate internationally, making it very hard for governments, Chinese or otherwise, to track them down and bring them to justice.
The risk to China is that continued exposure to deepfake scamming erodes the trust in technology that’s been such a large part of the country’s development. Artificial intelligence and its applications represent a huge opportunity for China’s future. The market is currently valued at RMB 700 billion (about US$97.5 billion) and predicted to grow by 34% before 2033.
Ultimately, the challenge is not just technical or legal, but also social. As deepfake technology continues to advance, the country faces a delicate balancing act – embracing the benefits of AI while protecting individuals and businesses from its misuse. Only by addressing both the technological and human dimensions of this threat can China ensure that its digital future remains both innovative and secure.
