The company are using DeepText to filter out ‘mean’ comments, and one Kanye line in particular is tripping it up
In a recent Wired profile, Instagram’s CEO and cofounder Kevin Systrom discussed his plans to clean up the app’s comments. The platform has been using DeepText, a word categorisation system built by Facebook, to separate ‘good’ text from ‘bad’ text. He made the decision to clean up Instagram and eliminate spam and hateful comments in June of last year, initially using human moderators and testing spam and abuse filters on an unlikely test case – Taylor Swift and the army of snake emojis that decorated her page in July of last year.
It worked well enough, but then in June this year, Systrom began using DeepText to assess the context of potentially abusive posts, some of which are often not that straightforward. DeepText has struggled with words that have different meanings across cultures and contexts. Furthering the Swift connection, Instagram has been testing DeepText’s abilities by feeding it both clean and explicit words and phrases – including Kanye West lyrics. It had some difficulty distinguishing the lyrics from insults, for example, every line in this controversial sequence in “Famous”: “for all my southside n----s that know me best/I feel like me and Taylor might still have sex/why? I made that bitch famous”.
DeepText is advanced, but still struggling with distinctions that humans would find it easier to make. However, Instagram now gives you the option to input words that you want to be filtered out of your comments – so if there’s a particular word or emoji people use to harm you that DeepText wouldn’t pick up as offensive, you can still block it. You can read Wired’s entire story on how Systrom is making Instagram nice again here.