MIT researchers made first attempts to gain a better understanding of deeplearning networks

MIT researchers made first attempts to gain a better understanding of deeplearning networks

28. Juli 2020 0 Von Horst Buchwald

MIT researchers made first attempts to gain a better understanding of deeplearning networks

New York, July 27, 2020

Deep learning systems make language recognition just as possible as autonomous driving. But how deep learning comes to these results has not yet been clarified. A group of MIT researchers recently reviewed their contributions with the aim of gaining a better theoretical understanding of deep learning networks.
„Deep learning was in some ways a chance discovery,“ said Tommy Poggio, researcher at the McGovern Institute for Brain Research, director of the Center for Brains, Minds, and Machines (CBMM) and Eugene McDermott Professor of Brain and Cognitive Sciences. „We still don’t understand why it works.“ But a theoretical framework is slowly taking shape. That’s why he believes “ we are now very close to a satisfactory theory“.
He says that the current period in human history is characterized by an overabundance of data – data from sensors of all kinds, text, the Internet and large amounts of genomic data generated in the life sciences. Computers today take these multidimensional data sets and generate a number of problems that the late mathematician Richard Bellman called the „curse of dimensionality“.
One of these problems is that the representation of a smooth, high-dimensional function requires an astronomically large number of parameters. It is well known that deep neural networks learn particularly well how to represent such complex data. But why is this so?
„Deep learning is like electricity after Volta discovered the battery, but before Maxwell,“ explains Poggio, who is The Core’s founding scientific advisor, MIT Quest for Intelligence, and an investigator in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT. „Useful applications were certainly possible after Volta, but it was Maxwell’s theory of electromagnetism, this deeper understanding, that then opened the way to radio, television, radar, transistor, computers and the Internet.
The theoretical treatment of Poggio, Andrzej Banburski and Qianli Liao provides clues as to why deep learning could overcome data problems like „the curse of dimensionality“. Their approach begins with the observation that many natural structures have a hierarchy. In order to model the growth and development of a tree, we would not need to specify the location of each branch. Instead, a model could use local rules to control branching hierarchically. The primate’s visual system seems to do something similar when processing complex data. When we look at natural images – including trees, cats and faces – the brain integrates local image fields one after the other, then small collections of fields, and then collections of collections of fields.
For Qianli Lao, author of the study and PhD student at the Department of Electrical Engineering and Computer Sciences and a member of the CBMM, „the physical world is compositional – in other words, it is composed of many local physical interactions,“ he explains. This goes „beyond images. Language and our thoughts are compositional, and even our nervous system is compositional when it comes to understanding how neurons connect.