In the late ’90s, Tomi Poutanen, a precocious computer whiz from Finland, hoped to do his dissertation on neural networks, a scientific method aimed at teaching computers to act and think like humans. As a student at the University of Toronto, it was a logical choice. Geoffrey Hinton, the godfather of neural network research, taught and ran a research lab there.
But instead of encouraging Poutanen, who went on to work at Yahoo and recently co-founded media startup Milq, one of his professors sent a stern warning about taking the academic path known as deep learning.
“Smart scientists,” his professor cautioned, “go there to see their careers end.” Hinton’s lab was seen as a renegade project, more the stuff of science fiction than vocation.
Now, a couple decades later, scientists are racing to start their careers in the field. Once passé, deep learning, the subset of artificial intelligence focused on teaching machines to find and classify patterns in mass quantities of data, is now de rigueur across Google, Facebook, Microsoft, IBM and a host of other Silicon Valley companies. The trend has ignited an expensive race to scoop up scarce talent. And much of that expertise ties back to a cabal-like group of researchers who kept the futuristic field on life support 15 years prior.
In the decades since Poutanen entered Toronto, deep learning fell into what’s often called an “AI Winter” — a period, typical of the ambitious community, where the promise of theory fails to meet practical applications. Financial support dries up and researchers lose interest. Scientists had developed advanced theories of how neural networks operate, but lacked the computing power and data to put them to work.
Three computer scientists, Hinton, Yann LeCun and Yoshua Bengio, apparently missed the memo.
They toiled away at their own labs and at a research institute in Toronto called CIFAR, chiseling away at the abstract computational methods. The trio jokingly referred to themselves as the “deep learning conspiracy.” Others called them the “Canadian Mafia.”
Their bet on the tech has paid off handsomely. In 2013, Hinton was hired as a distinguished researcher at Google, where he works on its expanding deep learning division; LeCun was tapped to lead Facebook’s AI efforts later that year; and last week, IBM announced it was working with Bengio, a professor at the University of Montreal, to infuse Watson, its super-computer, with deep learning. Re/code spoke with LeCun, Bengio and a bevy of experts in the field, many of whom pointed to the dogged work of the trio as the foundation for the next frontier in AI technology.
“In the lean times when no one believed in neural nets, these are the people who really kept the torch burning and really inspired a lot of people,” explained Rob Fergus, a former LeCun colleague who followed him to Facebook.
Fruits of their efforts are already starting to appear in front of consumers, with deep learning woven into products like the new Google Photos app and in the facial recognition technology infused in Facebook’s new app, Moments. And Facebook is considering a personal assistant product within Messenger, technology that could lean on deep learning’s computing prowess, according to a report yesterday in The Information. (Facebook declined to comment.)
Disciples of the three researchers are benefiting, too, and many of those who aren’t hired by bigger players out of grad school are gobbled up as part of acquisitions. Twitter absorbed numerous former students of the three researchers with its two recent AI acquisitions. Among the ranks of DeepMind, the secretive AI company that Google bought last year, are several of the Canadian Mafia’s adherents.
The interest in deep learning is similar to when “big data” was fashionable inside tech circles not so long ago. In many ways, this is its next iteration. Conversations with more than a dozen AI experts suggest that deep learning could soon be the backbone of many tech products that we use every single day.
Continue reading article – HERE