How AI tools fuel child exploitation on Malaysian social media
Warning: This article contains sensitive content about child sexual exploitation.
## Following the publication of this story, on July 9, 2026, the MCMC requested that Malaysiakini include these sentences.
On Oct 7, 2025, TikTok updated its community guidelines to further stamp out child sexual abuse materials (CSAM) by banning artificial intelligence-generated content that depicts minors in sexualised and seductive situations.
The United Nations agency for children, Unicef, has said AI-generated CSAM normalises the sexual exploitation of children and fuels demand for abusive content.
But a two-month Malaysiakini investigation into TikTok’s content moderation system found that despite a ban on the social media platform for such material, users are still able to obtain AI-generated content of minors in suggestive behaviour on the platform.
As a nonprofit journalism organization, we depend on your support to fund more than 170 reporting projects every year on critical global and local issues. Donate any amount today to become a Pulitzer Center Champion and receive exclusive benefits!
These include AI-generated videos and images of children and teenagers wearing swimsuits, bikinis, sexy lingerie, or the Japanese school uniform while dancing or running.
What is CSAM?
Under Malaysia’s Sexual Offences Against Children Act 2017 (SOACA), CSAM includes any visual, audio, or written depiction of a child or a person appearing to be a child engaged in sexually explicit conduct. The definition extends to real, simulated, and realistic digital depictions, including AI-generated images. This material was previously known as child pornography.
CSAM can be produced in various ways, including covert filming, the use of generative AI to sexualise images obtained without consent, or by coercing victims into creating sexual images through grooming, sextortion, or the exploitation of intimate relationships. Under the SOACA, the production, possession, distribution, sale, publication, access, solicitation, and sharing of CSAM are criminal offences.
While adults remain responsible for many CSAM offences, in some cases, children may also become involved in producing or distributing such material, particularly through the misuse of generative AI.
After collecting the data, Malaysiakini reported all 22 accounts via TikTok’s complaints system. This portal also filed formal complaints with the MCMC after receiving indifferent responses from the social media platform’s automatic content moderation system.
But will reporting obscene content suffice, given there are millions of TikTok accounts? What actions can social media platforms and authorities take when bad actors post AI-generated content that skirts the lines between what is acceptable and objectionable? And does TikTok’s own algorithm play a role in how such content is unintentionally distributed?
Innocent, youthful features and explicit seduction
The images in the 22 accounts are cut from the same cloth: innocent facial features paired with a curvaceous body that is dressed in attire that could be sexually suggestive.
Instead of adults, the models are teenagers or children. The content from all 22 accounts is focused on girls, except for two, which feature boys.
At first glance, some of the videos are seemingly proper; the AI avatars are all decently clothed, but once users hit play, the avatars’ actions immediately turn lewd and sexually suggestive.
Some of the disturbing images include AI-generated boys video calling while bouncing on a yoga ball, panting and blushing. To avoid promoting illicit content, this portal will refrain from sharing the details.
Some users who came across these accounts posted angry comments such as: “I know this is AI, but what are you doing?” Some even slammed the administrators for violating the law.
The investigation also came across some administrators of these accounts, who argued that they were permitted to republish their lewd content even after it was initially taken down by TikTok.
Monetising crime
Two of the accounts had enabled the “subscribe” feature, charging US$4 per month (RM16) for access. This feature, however, is unavailable to Malaysian TikTok account owners.
According to the social media tracking tool Social Insider, two of these accounts are based in the United States and Canada.
The TikTok Creator Academy indicates that creators can set their monthly subscription fees between US$2.99 and US$99.99.
TikTok claims on its official website that account holders will receive between 70 percent and 90 percent revenue share, depending on whether creators achieve a certain target.
A follow-up check on June 17 found that only one account remains active and continues posting AI-generated content.
Malaysiakini also found that at least six accounts embedded external links in their personal bio, channelling users to YouTube or other websites to browse more content, including detailed materials on how to produce AI-generated CSAM.
The popularisation of open-source and commercial AI tools means that anyone can easily acquire and master the technology to produce CSAM.
Shadows of generative tools: Sora and PixVerse
All of the accounts Malaysiakini analysed are labelled “AI-generated content” or contain hashtags like “#AI” in their description. However, most of the videos aren’t watermarked, making it impossible to identify which tools the offenders abused to produce such content.
Of the 22 accounts, two showed AI watermarks: OpenAI’s Sora, ByteDance’s Dreamina, and Beijing AISphere’s PixVerse.
Sora, Dreamina, PixVerse
Dreamina is a tool integrated into the video creation platform CapCut, which is owned by TikTok’s Chinese parent company ByteDance.
PixVerse is an AI-generated video platform with more than one million downloads on the Google Play Store, branding itself as a tool for users to generate movie-quality videos from prompts, photos, or clips, with no experience needed.
It’s reported that Alibaba Group, the Chinese e-commerce titan, has poured US$60 million into PixVerse, which announced that, up until September 2025, its user base reached 100 million globally.
Sora, owned by OpenAI, was shut down as of March 2026.
Malaysiakini was unable to determine the authenticity of certain watermarks as they can also be forged or maliciously added.
In response to Malaysiakini’s queries, PixVerse said it was unable to identify the specific content or account referenced in the enquiry following an internal review.
However, after examining the screenshots provided, the company noted watermarks and identifiers associated with multiple AI tools, suggesting the content may have been generated, edited, or processed using a combination of platforms rather than a single service.
PixVerse added that it has strict controls in place to prevent the creation of sexualised content involving minors.
“Any prompt, uploaded media, and/or generated media that combines a reference to minor age with sexually suggestive or explicit content is directly intercepted and blocked from generating any output,” PixVerse said.
“These measures are continuously reviewed and updated to address emerging forms of misuse. Upon receiving your report, we have initiated a review of our existing measures to identify any areas that require further strengthening.
Malaysiakini has contacted OpenAI for further clarification, but has not received any responses at the time of publication.
Report rejected in 30 minutes
TikTok has three content moderation checkpoints. The first one comprises an AI that will immediately detect whether a new post violates its community guidelines or copyrighted content.
In Malaysiakini’s report to TikTok’s complaints system on March 13, the 17 active accounts were flagged as “nudity or sexual activity,” and they were described as containing underage sexual intercourse, grooming, and abuse.
All 17 reports bounced back just after 30 minutes, with TikTok declaring that there were “no violations.” The related accounts and content remain intact even after 78 hours since the reports were submitted.
According to a Reuters report in October 2024, TikTok’s parent company, ByteDance, has cut hundreds of jobs globally, including in Malaysia, with the aim of using AI for content moderation.
When contacted by Malaysiakini, TikTok said accounts that violate its community guidelines are removed and that it continues to address evolving circumvention tactics.
TikTok did not respond directly to Malaysiakini’s queries on the 17 reports that this portal lodged through the social media platform’s complaints channel.
It also did not specify whether the 17 accounts identified by Malaysiakini had violated its policies or whether any action had been taken against them.
On March 16, Malaysiakini officially filed the same reports to MCMC, which took eight working days to complete its investigations, excluding the Hari Raya holidays.
The commission replied that it had forwarded the complaints to the platform and urged TikTok to further monitor the accounts to ensure they complied with community regulations. On April 3, MCMC declared the case “closed.”
## In response to this story, the MCMC told Malaysiakini that the complaint was filed in the reporter’s personal capacity and not by Malaysiakini.
## The commission also stated that the majority of complaints had been assessed by MCMC within two days of receipt and that it “independently assesses every complaint received in accordance with the applicable legal and regulatory framework and does not merely accept a complainant's assessment or characterisation of reported content.”
## MCMC also said that following its engagement with TikTok, most of the reported content was subsequently removed by the platform.
Legislative and online safety gap
Women’s Centre for Change Penang (WCC) programme director Karen Lai cautioned that content sexualising children can bring severe harm, as it normalises such behaviour and can lead to children internalising misconceptions while scrolling through social media.
"The threat is no longer hypothetical, but a reality that the law may not be prepared to address"
Karen Lai, Women’s Centre for Change Penang (WCC) programme director
When it comes to the law, Lai, a lawyer by training, explained that Section 4 of the SOACA defines CSAM to include: (a)(ii) a person who appears to be a child engaged in explicit sexual conduct; (a)(iii) a realistic or graphic depiction of a child engaged in explicit sexual conduct; (a)(iv) a realistic or graphic depiction of a person who appears to be a child engaged in explicit sexual conduct.
However, she added that these provisions were not necessarily drafted with generative AI in mind and arguably do not extend clearly to AI-generated images in which no “real” person or child is involved.
She added that although Section 233(3) of the Communications and Multimedia Act 1998 (CMA) prohibits the dissemination of obscene or indecent content to children and could theoretically be applied to AI-generated CSAM, enforcement may be hindered because the law does not explicitly include digitally generated content.
Suhakam said that the CMA and SOACA provide broad definitions of “content” and CSAM that could be interpreted to include AI-generated material.
However, it noted that “legal certainty can only truly be achieved through clear legislative amendments or court decisions,” a view echoed by Lai.
To close gaps in existing legislation, the Dewan Rakyat passed the Cybercrime Bill 2026, which is intended to complement — not replace — current laws. The bill introduces offences covering AI-generated or manipulated content used to facilitate crime, punishable by up to RM500,000 in fines and/or seven years’ imprisonment.
Lawyer Lo Khai Yi is placing his hopes on the Online Safety Act 2025 (ONSA), which will be implemented this year.
Under the act, platforms with eight million users or more are required to register for a licence in Malaysia, set up a harmful content disposal mechanism, and a reporting portal to enhance child safety protection. Licence holders who fail to comply can be fined up to RM 10 million.
On March 3, in a written reply to Parliament, Communications Minister Fahmi Fadzil said that TikTok, along with Facebook, Instagram, WhatsApp, YouTube, and Telegram, have automatically been deemed as licence holders subject to ONSA’s requirements since Jan 1.
As an expert in technology and media law, Lo pointed out that the new law mandates licensed platforms to establish mechanisms to “prioritise the handling of harmful content” and strictly prohibit user access.
He believes that ONSA specifically enshrines CSAM as the top priority and that platforms are legally obligated to detect and remove this kind of harmful content, including that generated by AI.
Dark side of the algorithm
TikTok did not respond directly to questions about the above behaviour of its algorithm.
In a statement to Malaysiakini, TikTok said the company “prohibits harmful AI-generated content, including sexualised, fetishised, or victimising depictions.”
The TikTok algorithm experience described above mirrors findings from a November 2023 Amnesty International study, which found that the platform’s “For You” recommendation system is designed to maximise user engagement and time on the app, even if that content is potentially harmful.
The two-part Amnesty study detailed “abuses experienced by children and young people using TikTok, and how these abuses are caused by TikTok’s recommender system and the underlying business model.”
The Amnesty study did not specifically look at CSAM but tracked content that encouraged self-harm and suicide ideation.
“(The study) shows that TikTok’s very business model is inherently abusive and privileges engagement to keep users hooked on the platform, to collect evermore data about them,” the human rights group said.
“TikTok then uses this data to create profiles of users and draw inferences about them, which allows it to cluster users in groups to target them with highly personalised content to keep them engaged.
“To the extent that TikTok has put in place policies and practices to ensure greater respect for children’s rights, they differ from region to region, leaving children and young people in some parts of the world exposed to exploitative data collection in others.”
As the experts and Malaysiakini’s own investigation showed, suggestive and problematic AI-generated content seems to exist in a grey area and flies under the radar of TikTok’s safeguards.
Coupled with an algorithm that prioritises retaining engagement and the shortcomings of the law, this grey area risks becoming an unregulated black hole for bad actors.
Need help?
If you or someone you know is experiencing online sexual exploitation, grooming, or abuse, please seek help and report it to the relevant authorities or support organisations.
Talian Kasih 15999: A 24/7 emergency helpline established by KPWKM and operated by Jabatan Kebajikan Masyarakat (JKM)
Internet Watch Foundation (IWF): Report online child sexual abuse material: https://report.iwf.org.uk/my
NCMEC CyberTipline: https://report.cybertip.org/
MCMC Complaint Portal: https://aduan.mcmc.gov.my/
