Twitch plans to test an AI experiment to detect offensive and disrespectful chat messages

Nowadays, every major social media platform on the internet incorporates artificial intelligence at least to some degree. Whether it be AI-generated adaptive thumbnails on YouTube, post summaries on X, or automatic picture enhancements on Instagram, companies are using it to improve user experiences, and now Twitch has introduced a new feature.

During its growth, Twitch consistently saw problems with chat moderation for several reasons. First off, the scalability of moderation tools has been problematic, particularly with the older Internet Relay Chat (IRC) bots that can no longer manage large, multi-channel audiences effectively.

In a 2019 YouTube video titled "Twitch Mods Have Created a Platform of Fear and Toxicity," streamer Charlie "MoistCr1TiKaL" highlighted concerns about inconsistent and non-transparent enforcement of the platform's policies. This led to problems, especially among smaller creators.

Streamer and political commentator, HasanAbi, has also faced issues with the platform regarding chat bans.

While the platform has introduced tools like AutoMod to help moderators, the complexity of these tools still places a heavy burden on human moderation, especially during large streams.

AutoMod was introduced in 2016, and although it was useful in censoring some profanity, other hateful comments would sometimes pass through the radar (Image via @TwitchSupport/X)

Fast forward to the current day, the website now plans to incorporate its take on AI moderation, this time within the chatboxes.

---

Twitch will reportedly conduct an AI experiment to help filter hateful messages in chatboxes

Now, the website plans to test a feature where potentially harmful messages will be temporarily held before being posted in chat. Users will then be required to confirm their intent to send it.

This new feature could lead to slower message delivery, especially during fast-paced streams. However, this delay might significantly enhance moderation by reducing the number of harmful or impulsive messages sent, giving users time to reconsider using toxic language.

In the website's history, quite a few creators have been banned, often due to inappropriate behavior, which is then amplified through the chatbox. An example of this would be when Kick streamer Adin Ross was banned from the platform for hateful conduct. His chat had a role to play in this as well.

---

Twitch is not the first streaming platform to implement AI-based chat moderation, Kick has also taken steps to enable creators to better control the messages their streams receive.

Creators can filter through redundant, irrelevant, distasteful, and inappropriate messages using the AI filter option on the creator dashboard in the settings.