Sources Contact Advanced Search Tutorials

An Interest In:

Web News this Week

Search Archive

Some of Our Sources

View All Sources

Help Webnuz

Referal links:

November 3, 2020 05:24 pm

AI Godfather Geoff Hinton: "Deep Learning is Going To Be Able To Do Everything"

An excerpt from MIT Technology Review's interview with Geoffrey Hinton: You think deep learning will be enough to replicate all of human intelligence. What makes you so sure? I do believe deep learning is going to be able to do everything, but I do think there's going to have to be quite a few conceptual breakthroughs. For example, in 2017 Ashish Vaswani et al. introduced transformers, which derive really good vectors representing word meanings. It was a conceptual breakthrough. It's now used in almost all the very best natural-language processing. We're going to need a bunch more breakthroughs like that. And if we have those breakthroughs, will we be able to approximate all human intelligence through deep learning? Yes. Particularly breakthroughs to do with how you get big vectors of neural activity to implement things like reason. But we also need a massive increase in scale. The human brain has about 100 trillion parameters, or synapses. What we now call a really big model, like GPT-3, has 175 billion. It's a thousand times smaller than the brain. GPT-3 can now generate pretty plausible-looking text, and it's still tiny compared to the brain. When you say scale, do you mean bigger neural networks, more data, or both? Both. There's a sort of discrepancy between what happens in computer science and what happens with people. People have a huge amount of parameters compared with the amount of data they're getting. Neural nets are surprisingly good at dealing with a rather small amount of data, with a huge numbers of parameters, but people are even better. A lot of the people in the field believe that common sense is the next big capability to tackle. Do you agree? I agree that that's one of the very important things. I also think motor control is very important, and deep neural nets are now getting good at that. In particular, some recent work at Google has shown that you can do fine motor control and combine that with language, so that you can open a drawer and take out a block, and the system can tell you in natural language what it's doing. For things like GPT-3, which generates this wonderful text, it's clear it must understand a lot to generate that text, but it's not quite clear how much it understands. But if something opens the drawer and takes out a block and says, "I just opened a drawer and took out a block," it's hard to say it doesn't understand what it's doing.

Slashdot

Slashdot was originally created in September of 1997 by Rob "CmdrTaco" Malda. Today it is owned by Geeknet, Inc..

More About this Source Visit Slashdot