How Google is fighting the war on internet trolls
The internet can be a harsh place. It seems like for every feel-good story or picture of a puppy playing with a kitten, there are 1,000 trolls rummaging through the depths of their minds to post the most vile comments they can imagine. And if you’re a woman or person of color, well, multiply that troll army by 10.
But hey, that’s the internet, right? Except it doesn’t have to be that way. And it might not be for much longer if the folks at Google (GOOG, GOOGL) subsidiary Jigsaw have their way. A kind of high-powered startup inside Google’s parent company Alphabet, Jigsaw focuses on how technology can defend international human rights.
The toxicity of trolls
The company’s latest effort is called the Perspective API. Available Thursday, Feb. 23, Perspective is the result of Jigsaw’s Conversation AI project and uses Google’s machine learning technologies to provide online publishers with a tool that can automatically rank comments in their forums and comments sections based on the likelihood that they will cause someone to leave a conversation. Jigsaw refers to this as a “toxicity” ranking.
“At its core, Perspective is a tool that simply takes a comment and returns back this score from 0 to 100 based on how similar it is to things that other people have said that are toxic,” explained product manager CJ Adams.
Jigsaw doesn’t become the arbiter of what commenters can and can’t say in a publisher’s comment section, though. Perspective is only a tool that publishers use as they see fit. For example, they can give their readers the ability to filter comments based on their toxicity level, so they’ll only see non-toxic posts. Or the publisher could provide a kind of feedback mechanism that tells you if your comments are toxic.
The tool won’t stop you from submitting toxic comments, but it will provide you with the nudge to rethink what you’re writing.
Perspective isn’t just a bad word filter, though. Google’s machine learning actually gives the tool the ability to understand context. So it will eventually be able to tell the difference between telling someone a vacuum cleaner can really suck and that they suck at life.
Perspective still makes mistakes, as I witnessed during a brief demo. But the more comments and information it’s fed, the more it can learn about how to better understand the nuances of human communication.
Jigsaw’s global efforts
In its little over a year of existence, Jigsaw has implemented a series of projects designed to improve the lives of internet users around the world. Project Shield, for example, is a free service that protects news sites from distributed denial of service (DDoS) attacks. Redirect Method uses Adwords targeting tools to help refute ISIS’ online recruitment messages, while Montage helps researchers sort through thousands of hours of YouTube videos to find evidence of potential war crimes.
“We wake up and come to work everyday to try to find ways to use technology to make people around the world safer,” Jigsaw President Jared Cohen said. “We are at this nexus between international security and business.”
Cohen said Jigsaw’s engineers travel around the world to meet with internet users vulnerable to harassment and other online-based rights abuses, such as individuals promoting free speech or opposing authoritarian regimes, to understand their unique challenges. And one of the biggest problems, Cohen explained, has been online harassment.
Trolls aren’t always just cruel
Dealing with trolls is par for the course in the US. But in other countries, harassment in comment sections and forums can have political implications.
“In lots of parts of the world where we spend time [harassment] takes on a political motivation, sectarian motivation, ethnic motivation and it’s all sort kind of heightened and exacerbated,” Cohen explained.
But with Perspective, Jigsaw can start to cut down on those forms of harassing comments, and bring more people into online conversations.
“Our goal is to get as many people to rejoin conversations as possible and also to get people who everyday are sort of entering the gauntlet of toxicity to have an opportunity to see that environment improve,” said Cohen.
The path to a better internet?
Jigsaw is already working with The New York Times and Wikipedia to improve their commenting systems. At The New York Times, the Perspective API is being used to let The Gray Lady enable more commenting sections on its articles.
Prior to using Perspective, The Times relied on employees to manually read and filter comments from the paper’s online articles. As a result, just 10% of stories could have comments activated. The Times is using Perspective to create an open source tool that will help reviewers run through comments more quickly and open up a larger number of stories to comments.
Wikipedia, meanwhile, has been using Perspective to detect personal attacks on its volunteer editors, something Jigsaw and the online encyclopedia recently published a paper on.
With the release of Perspective, publishers and developers around the world can take advantage of Google technologies to improve their users’ experiences. And the conversation filtering won’t just stop hateful comments. Cohen said the company is also working to provide publishers and their readers with the ability to filter out comments that are off-topic or generally don’t contribute to conversations.
If Perspective takes off, and a number of publications end up using the technology, the internet could one day have far fewer trolls lurking in its midst.
More from Dan:
Dell XPS 13 2-in-1 review: The best laptop around learns a new trick
Symantec CEO: ‘It’s a new theater of war’ for cybercriminals
Email Daniel at [email protected]; follow him on Twitter at @DanielHowley.