Our mushy brains seem a far cry from the solid silicon chips in computer processors, but scientists have a long history of comparing the two. As Alan Turing put it in 1952: “We are not interested in the fact that the brain has the consistency of cold porridge.” In other words, the medium doesn’t matter, only the computational ability.
Deep learning is the type of machine-learning that powers today’s most advanced artificial intelligence systems. Deep neural networks are algorithms that process massive amounts of data using hidden layers of interconnected, interconnected nodes. Deep neural networks, as their name implies, were inspired from real brain neural networks. The nodes are modeled after actual neurons or, at the very least, what neuroscientists learned about neuronal networks back in 1950s when the influential model of the perceptron emerged. Our understanding of how single neurons work has greatly improved, and biological neurons have been shown to be much more complicated than artificial neurons. How much, you ask?
David Beniaguev (external-link), Idan Segev (data-event click=”” data-offer URL=”https://www.sciencedirect.com/science/article/abs/pii/S0896627321005018″) and Michael London (all at the Hebrew University of Jerusalem) trained an artificial deep neural net to replicate the calculations of a biological neuron. They showed that a deep neural network requires between five and eight layers of interconnected “neurons” to represent the complexity of one single biological neuron.
These complications were not anticipated by even the authors. Beniaguev said, “I expected it to be simpler and more compact.” Beniaguev expected three to four layers to be sufficient for the capture of the calculations within each cell.
Timothy Lillicrap, who designs decisionmaking algorithms at the Google-owned AI company DeepMind, said the new result suggests that it might be necessary to rethink the old tradition of loosely comparing a neuron in the brain to a neuron in the context of machine learning. He said that the paper “really helps to force you to think about this more thoroughly and examine how far you can draw those analogies.”
How they deal with incoming information is the most fundamental analogy between real and artificial neurons. Each type of neuron receives incoming information and decides whether or not to transmit it to others. Although artificial neurons can make these decisions using a straightforward calculation, research over decades has shown that biological neurons are far more complex. To model the interaction between inputs from a neuron’s tree-like branches and its decision to emit a signal, computational neuroscientists employ an input-output function.
The new research taught the artificial deep neural network how to mimic this function in order to establish its complexity. From a rat’s cortex, they created a huge simulation of the input/output function for a particular type of neuron that had distinct branches of dendritic branch at its top, and bottom. This neuron is called a pyramidal neural network. They then fed this simulation into deep neural networks that contained up to 256 artificial neuronal layers. The number of layers was increased until the simulation reached 99 percent accuracy between input and output. With at most five artificial layers, but not more than eight, the deep neural network correctly predicted the neuron’s input and output functions. This equated roughly to approximately 1,000 artificial neurons per biological neuron in most networks.
“[The result] forms a bridge from biological neurons to artificial neurons,” said Andreas Tolias, a computational neuroscientist at Baylor College of Medicine.
However, the authors of this study cautioned that it is not yet a simple correspondence. London stated that the relationship between the number of layers in a neural network’s network and its complexity isn’t clear. We don’t know how many layers are added to a neural network by switching from four to five. We cannot say that 1,000 artificial neurons is equivalent to a biological neuron that’s exactly 1000 times more complex. It is possible to use exponentially more artificial neuronal layers within each layer to eventually create a deep neural network. However, this would require more data and more time to train the algorithm.
London said, “We tried many different architectures, with many depths, and many things,” To encourage others to discover clever solutions with less layers, the authors shared their code. The authors believe that the result provides a useful comparison to further research, despite the difficulty of finding a deep neural network capable of imitating the neuron at 99 percent accuracy. Lillicrap suggests it could be a way to link image classification networks to the brain, often requiring upwards to 50 layers. A biological neuron can be compared to a five-layer artificial neural system. An image classification network that has 50 layers could then be equivalent to 10 biological neurons.
Their results will also change AI’s current state-of the-art deep network architecture, according to the authors. Segev stated that the Deep Network Technology should be replaced to bring it closer to the human brain. “A unit representing a neuron is a basic unit of deep network technology, and each unit within the deep network must be replaced with one which represents the individual. This will allow the network to become more like the brain.” To replace each artificial neuron, AI engineers and researchers could use a 5-layer deep network to create a “mini-network” for this purpose.
Some question whether AI would benefit from this. “I think that’s an open question, whether there’s an actual computational advantage,” said Anthony Zador, a neuroscientist at Cold Spring Harbor Laboratory. This work provides the basis for testing this.”
Outside of AI applications, the new paper also adds to a growing consensus on the strong computational power of dendritic trees and, by proxy, single neurons. Back in 2003, a trio of neuroscientists showed that the dendritic trees of a pyramidal neuron perform complex computations by modeling it as a two-layer artificial neural network. The new paper examines the factors that led to the greater complexity of their deep neural networks, which are five- to eight layers thick. The dendritic tree and a receptor on the dendrites, which receives chemical messengers from them led to the discovery. These findings were consistent with other previous research in this field.
Many believe that the results suggest that single-neuron research should be given more priority by neuroscientists. “This paper makes thinking about dendrites and individual neurons much more important than it was before,” said Konrad Kording, a computational neuroscientist at the University of Pennsylvania. Zador and Lillicrap suggested it would be equally important to focus on the neurons in a particular circuit for understanding how single neurons are used.
The language of artificial neural network may offer new insights into how neurons work and ultimately the brain. “Thinking in terms of layers and depths and widths gives us an intuitive sense of computational complexity,” said Grace Lindsay, a computational neuroscientist at University College London. Lindsay warns, however that this new research is only comparing an existing model to another model. It’s not possible for neuroscientists at the moment to capture the entire input-output function a neuron. Real neurons could be more complicated.
London stated that “we aren’t sure that between 5 and 8 is the ultimate number.”
Reprinted original story with permission of Quanta Magazine. This independent, editorially-controlled publication is part of the Simons Foundation and aims to increase public knowledge of science. It covers research trends and developments in math and other life and physical sciences.
- The latest on tech, science, and more: Get our newsletters!
- Rain boots, turning tides, and the search for a missing boy
- Better data on ivermectin is finally on the way
- A bad solar storm could cause an “internet apocalypse”
- New York City wasn’t built for 21st-century storms
- 9 PC games you can play forever
- Explore AI like never before with our new database
- WIRED Games: Get the latest tips, reviews, and more
- Do you want the best health tools? Our Gear team has compiled a list of the top running and fitness gear, including shoes and socks.
Publiated at Sun, 12 Sep 2021 12:00:43 +0000