The algorithm and user wellbeing
The European Commission issued a preliminary ruling on Friday charging TikTok with failing to protect users’ mental health. I find the Commission’s direct attack on the platform’s engagement-based recommender system a necessary step to prevent the "rabbit-hole effects" that trap vulnerable users. The regulator claims TikTok’s algorithm can be harmful by serving a continuous stream of personalised videos that risk worsening anxiety or eating disorders.
Scores of videos on a single theme can create a cumulative effect that damages users’ mental health by pushing them into repetitive loops, even if one video alone is not problematic. TikTok failed to prevent users from being served repetitive content on themes like body image or isolation. Henna Virkkunen, the European Commission’s lead official on tech policy, said TikTok needs to step up efforts to protect users’ wellbeing, particularly for minors. Every time a user scrolls or likes a video, the algorithm serves more of the same content. This process targets 189 million users across the EU. The investigation targets content that can lead to sadness or exacerbate existing eating disorders.
TikTok’s safeguards to mitigate these risks proved ineffective. If a user selects a "not interested" button on a video about eating patterns, the algorithm still offers similar content on diet and body image. A feed refresh only provides a brief respite from a stream of content on one topic. This failure makes the "not interested" option feel like an afterthought. I view the Commission’s focus on these ineffective measures as the most damning part of the report.
Engagement models and the DSA
The Commission’s investigation targets the platform’s core business model, which focuses on keeping users scrolling for as long as possible through an engagement-based recommender system that can lead users down rabbit holes. Only 0.2% of TikTok users actually select the non-personalized feed option provided by the Digital Services Act. This engagement-based model prioritizes interaction over user safety. One Commission official, speaking with anonymity, said this marks the first time the Commission has really tackled the engagement-based recommender system model head-on. The official explained that remedies must be effective and that TikTok’s current measures fail to work.
| Metric | Detail |
|---|---|
| TikTok EU users | 189 million |
| Non-personalized feed users | 0.2% |
| Max potential fine | 6% of global annual turnover |
| Investigation start date | February 2024 |
Commission officials concentrated on mental health impacts because the evidence was strongest in this area. While the investigation could address radicalization, they focused on mental health. This fifth case of the EU executive accusing TikTok of breaching rules targets the specific way the platform’s design can create risks for minors and vulnerable adults. A Commission official told reporters that an engagement-focused business model can potentially create a risk.
Legal battles and possible fines
TikTok can now examine the investigation file to mount a defence or propose changes to its recommender system to avoid a fine. The company previously fought findings regarding its "addictive design" for features like infinite scroll, calling the Commission’s preliminary findings a "categorically false and entirely meritless depiction of our platform." Last December, TikTok made binding commitments to change its policies following a complaint regarding advertising transparency. TikTok can examine the evidence and defend its practices to avoid these penalties.
The current investigation follows the February 2024 launch under the Digital Services Act, which requires large internet companies to prevent harms such as disinformation and damage to mental health. TikTok faces a fine of up to 6% of its global annual turnover if the EU officials uphold their finding. To avoid this, TikTok might change its ForYou feed to give users more control. One Commission official noted that "not interested" buttons must have an effective meaning rather than being an afterthought that leads back to the old recommender system. TikTok can examine the evidence and defend its practices to avoid these penalties.
