How A Security Company Learned To Recognize The Sound Of Fraud

Someday soon, you may be able to unlock your car or even log in to your favorite streaming app on a hotel TV simply with the sound of your voice.

Pindrop, an Atlanta company that now primarily offers sound-based fraud detection tools for call centers, plans to release a service later this year that will let connected devices verify who they’re talking to, turning the human voice into a combination of a username and password.

“Everybody has a unique voice, and everybody has a unique behavior in the way they say things,” says Pindrop cofounder and CEO Vijay A. Balasubramaniyan.

Secure voice recognition login could make it safer to conduct complex transactions through digital assistants like Apple’s Siri and Amazon’s Alexa—and it could prevent scenarios in which loud TV advertisements or children playing with devices accidentally give commands (and buy products) via such devices. And if different devices adopted Pindrop or another common voiceprint provider, users wouldn’t have to separately program each device to recognize their individual speech patterns and could log in to new devices simply by speaking, says Balasubramaniyan.

The technology should also be able to detect when people are distorting their voices or playing recordings of other people speaking—fraud techniques he says they already encounter in the call center market, where Pindrop reports it works with eight of the top ten U.S. banks and two of the top five insurers to detect phone scams. Call center customers pay based on call volume, and Pindrop will likely roll out similar cost structures for IoT device makers he says.

For phone fraud detection, Pindrop’s systems don’t just listen to the sound of callers’ voices as they dial in to place orders or transfer funds—it also uses other audio data to determine where people are calling from and the type of phones and networks they’re using. Different models of phones introduce their own acoustic signatures into conversations, Balasubramaniyan says. And phone networks in different parts of the world transmit different sound frequency ranges based on different requirements for balancing bandwidth consumption and voice quality. Even internet calling tools, like Skype and Google Voice, break conversations into different-sized audio packets, making it possible for Pindrop’s algorithms to tell them apart.

“Every time you drop a packet, you’re having a break of 30 milliseconds as opposed to 20 milliseconds or whatever else,” Balasubramaniyan says.

That lets the company alert its clients when calls with strange characteristics come in, like if a number registered to a U.S. cellphone carrier is actually dialing through Skype or through a telephone network in Nigeria. In other cases, the company has detected calls ostensibly from different customers and numbers all being placed by the same fraud ring, he says.

“We’ll see, for example, hundreds of calls trying to access hundreds of accounts, but they all come [with] the same 147 characteristics, and it’s actually the same device,” he says.

Pindrop generates numeric and color-coded alert levels for its clients, letting them serve trustworthy callers more quickly and handle potential imposters with more care. Some might ask additional verification questions of users dialing in from unusual connections or even ask to call them back at their numbers on file. And others use the Pindrop score information simply for offline processing, deciding whether to approve transactions like wire transfers based on the indicated risk level.

While fraudsters do try new techniques over time, Balasubramaniyan says Pindrop is often still able to pick up on new patterns of behavior—like routing calls through hacked telephone systems in less suspicious countries—based on the new acoustic signals they generate.

“It looks even more different than regular traffic,” says Balasubramaniyan.

The company can also detect known criminals connecting through new techniques based on their historic behavior patterns and voice data.

“We have created the world’s largest database of well-known voice fraudsters,” he says.

The company even operates a digital call center of its own, buying from carriers the phone numbers that their customers have traded in due to excessive numbers of fraudulent calls. Pindrop’s system receives about 90,000 fraudulent calls per day across those numbers, generally playing randomized recordings of phrases like “I can’t hear you very clearly” just long enough to keep callers on the line and fingerprint their connections.

All of the human voice data the company has captured will likely help make its IoT product more robust in preventing fraud, Balasubramaniyan says.

“Right now, we already see people who distort their voices,” he says. “Just knowing how voices sound over a massive dataset, that allows us to use that knowledge to also do this better.”