New Facebook artificial intelligence will 'mediate' your online activity

© Getty Images

Let's say you're out drinking with your buddies, things get out of hand, you pull out your smartphone, you take a selfie in the middle of all this drunken revelry, then you take 30 or 40 more, and, without hesitation, you start uploading them to Facebook.

It's a common thing to do. But Yann LeCun aims to stop such unbridled behavior - or at least warn people when they're about to do something they might regret. He wants to build a kind of Facebook digital assistant that will, say, recognize when you're uploading an embarrassingly candid photo of your late-night antics. In a virtual way, he explains, this assistant would tap you on the shoulder and say: "Uh, this is being posted publicly. Are you sure you want your boss and your mother to see this?"

The idea is more than just an idle suggestion. LeCun is the New York University researcher and machine-learning guru who now oversees the Facebook Artificial Intelligence Research lab, a team of AI researchers inside the internet giant that spans offices in both California and New York, and this rapidly expanding operation is now laying the basic groundwork for his digital assistant.

Fashioning such a tool is largely about building image recognition technology that can distinguish between your drunken self and your sober self, and using a red-hot form of artificial intelligence called "deep learning" - a technology bootstrapped by LeCun and other academics - Facebook has already reached a point where it can identify your face and your friends' faces in the photos you post to its social network, letting you more easily tag them with the right names.

'Uh, This Is Being Posted Publicly. Are You Sure You Want Your Boss And Mother To See This?

Today marks the one year anniversary of LeCun's Facebook lab - known as FAIR inside the company - and its research is powering the world's largest social network in more ways than one. The team's deep learning algorithms now examine your overall Facebook behavior in an effort to identify the right content for your news feed - content you're likely to click on - and they'll soon analyze the text you type into status posts, automatically suggesting relevant hashtags. But LeCun and his team are also looking towards AI systems that can understand Facebook data in more complex ways - and guide you in directions you may not go on your own.

"Imagine that you had an intelligent digital assistant which would mediate your interaction with your friends," he says, "and also with content on Facebook."

For some, this is a harrowing proposition. They don't want machines telling them what to do, and they don't want machines identifying their faces and storing them in some distant data center where they can help Facebook, say, target ads. But for LeCun, FAIR's work is about giving you more control over your online identity, not less. He also envisions a Facebook that instantly notifies you when someone you don't know posts your photo to the social network without your approval. "You will have a single point of contact to mediate your interaction but also to protect your private information," he says.

He and his Facebook team are by no means alone. Their work is part of a much larger movement towards deep learning, which seeks to automate online tasks by mimicking the behavior of the massive networks of neurons in the human brain. Taping the power of hundreds or even thousands of computers, Google uses deep learning to hone its search engine, recognize the commands you speak into your Android phone, and identify images on its Google+ social network. Microsoft uses it to translate Skype calls from one language to another. And everyone from Twitter to Yahoo is following suit.

The technology has become so important to the internet's biggest names that we're seeing a kind of arms race for deep-learning talent. Google snapped up Geoff Hinton, the University of Toronto professor who founded the deep learning movement alongside LeCun and others. Chinese search giant Baidu recently nabbed Andrew Ng, who helped found the deep learning program at Google. And since he was hired last year to run FAIR, LeCun has stolen some notable names from the Mountain View search giant, including Jason Weston and Tomas Mikolov.

The Power of Language

Deep learning isn't really a new technology. LeCun, Hinton, and others have explored the basic concepts since the'80s, and according to John Platt, a longtime research at Microsoft, the software giant was using similar techniques to provide handwriting recognition on tablet PCs a good ten years ago. But as Platt points out, thanks to recent advances in computer hardware - and the internet's ability to generate the massive amounts of data needed to help train neural nets - the technology has recently taken off in enormous ways.

Across the industry, it's already reinventing image and speech recognition. But like Google, LeCun and FAIR are pushing for more. The next big frontier, he says, is natural language processing, which seeks to give machines the power to understand not just individual words but entire sentences and paragraphs.

Before coming to Facebook, Mikolov led the creation of a deep learning system called Word2Vec, which aims to determine the particular relationships between words, and Google says this was used to improve its "knowledge graph," the system that helps the company's search engine map all those complex connections among websites. Now, he and Weston have brought this kind of expertise to the Facebook lab.

In the short term, LeCun explains, Facebook aims to create systems that can automatically answer simple questions. The company recently demonstrated a tool that can ingest a summary of The Lord of The Rings and then answer questions about the books. And it's exploring a kind of artificial short-term-memory that seeks to improve translation systems that use what are called "recurrent neural nets." Just as you can think of a neural net as the cerebral cortex that handles the translation itself, he says, his team is building a system akin to the hippocampus that can serve as "scratch pad" memory for that cortex.

'An AI-Complete Problem'

The larger aim, LeCun says, to create things like his digital assistant, things that can closely analyze not only photos but all sorts of other stuff posted to Facebook. "You need a machine to really understand content and understand people and be able to hold all that data," he says. "That is an AI-complete problem."

But at the same time, the team is looking beyond this sort of thing, hoping to anticipate that ways that Facebook will evolve in the more distant future - five or ten years down the road. LeCun hints this might involve the Oculus Rift - the virtual reality headset that Facebook acquired earlier this year - saying his team has at least discussed research with the Oculus team.

Certainly, there are limits to the company's AI ambitions. At one point, LeCun indicates that Facebook is not yet exploring AI in combination with robotics. But he does say this is something he's interested in exploring with his academic research, under the aegis of NYU. It's the next logical step.

Want something else to read? How about 'Grievous Censorship' By The Guardian: Israel, Gaza And The Termination Of Nafeez Ahmed's Blog