IBM’s Project Debater is an AI that’s ready to argue

Project Debater is the latest AI-based system from IBM’s research team (the folks behind Watson). The art of argument is a fundamental human experience. It’s how we govern, research controversial subjects, decide how we’ll live our lives and, more recently, how we spend much of our time on the internet. With a database of hundreds of millions of papers, reports and news articles, Project Debater is well-armed. But spouting facts and figures has very little to do with arguing. Instead, it’s taking that data and forming a cohesive narrative that wins debates and IBM’s system is impressive (but not quite up to par with humans).

IBM staged two debates (government-subsidized space exploration and the expansion of telemedicine) on curated topics that Project Debater was unaware of ahead of the event. The company wanted to make sure the system could present an argument without knowing what it would be researching. While its opening arguments were impressive, it was the system’s ability to create a narrative piecing together bits of knowledge and sentences from its vast treasure trove of papers after it ingested the speech of its opponent.

The system listened to four minutes of its human opponent’s opening remarks, then parsed that data and created an argument that highlighted and attempted to debunk information shared by the opposing side. That’s incredibly impressive because it has to understand not only the words but the context of those words. Parroting back Wikipedia entries is easy, taking data and creating a narrative that’s based not only on raw data but also takes into account what it’s just heard? That’s tough.

It’s so difficult, that during some of the system’s early tests it veered from the topic (teaching physical education in schools) towards something a bit more risque (teaching sex education in schools). It’s an easy leap without completely understanding the context of the conversation. Also, it was hilarious and uncomfortable.

While humans debating machines is a fun showcase of the technology, the real power of Project Debater will be its ability to present unbiased arguments, according to IBM researchers Noam Slonim and Ranit Aharonov.

“It’s a decision-making system can help break the echo chamber effect,” Aharonov told Engadget. “Instead of making a gut-reaction decision, it’ll help make unbiased decisions.”

In a world where emotion and bias colors all our decisions, Project Debater could help companies and governments see through the noise of our life experiences and produce mostly impartial conclusions. Of course, the data set it pulls from is based on what humans have written and those will have their own biases and emotion.

While the goal is an unbiased machine, during the discourse Project Debate wasn’t completely sterile. Amid its rebuttal against debater Dan Zafrir, while they argued about telemedicine expansion, the system stated that Zafrir had not told the truth during his opening statement about the increase in the use of telemedicine. In other words, it called him a liar.

When asked about the statement, Slonim said that the system has a confidence threshold during rebuttals. If it’s feeling very confident it creates a more complex statement. If it’s feeling less confident, the statement is less impressive.

So Project Debater either wasn’t confident or too confident about its rebuttal, so it essentially called Zafrir a liar, which is, sadly, the most human thing it did all night.