wild though With the success of ChatGPT and other major language models, the artificial neural network (ANN) that underpins these systems may be going in the wrong direction.
For one, ANN is “super-energy,” say Cornelia Fermüller, a computer scientist at the University of Maryland. “And the other problem is [their] lack of transparency.” Such systems are so complex that no one really understands what they’re doing or why they work so well.On the contrary, this makes them nearly impossible to deduce by analogy. , which is what humans do—use symbols for objects, ideas, and the relationships between them.
Such shortcomings may stem from the current structure of ANNs and their building blocks: individual artificial neurons. Each neuron receives input, performs computation, and generates output. Modern ANNs are complex networks of these compute units, trained to perform specific tasks.
However, the limitations of ANNs have long been apparent. For example, consider an ANN that distinguishes between circles and squares. One way to do that is to have two neurons in its output layer, one representing a circle and one representing a square. If you want your ANN to also distinguish the color of shapes—for example, blue or red—you will need four output neurons: one for a blue circle, a blue square , red circle and red square. More features means more neurons.
This cannot be how our brains perceive the natural world, with all its variations. “You have to propose that, you have one neuron for all combinations,” says Bruno Olshausen, a neuroscientist at the University of California, Berkeley. “So you will have in your brain, [say,] a purple Volkswagen detector.”
Instead, Olshausen and others argues that information in the brain is represented by the activity of many neurons. So the perception of a purple Volkswagen is not encoded into the action of a single neuron, but as the action of thousands of neurons. The same group of neurons, activated differently, could represent a completely different concept (perhaps a pink Cadillac).
This is the starting point for a completely different approach to computing, known as super-dimensional computing. It is important that each piece of information, such as the concept of an automobile or its make, model or color, or all of that information together, is represented as a single entity: a super dimensional vectors.
A vector is simply an ordered sequence of numbers. For example, a 3D vector consists of three numbers: number x, y, And z coordinates of a point in 3D space. A super-dimensional vector, or hypervector, can be an array of 10,000 numbers, representing a point in 10,000-dimensional space. These mathematical objects and the algebra to manipulate them are flexible and powerful enough to take modern computing beyond some of its current limitations and spur a new approach to artificial intelligence. .
“This is what I am most excited about, in fact, in my entire career,” Olshausen said. For him and many others, super-dimensional computing promises a new world in which powerful and efficient computing and machine-made decisions are completely transparent.
Step into the height space
To understand how super vectors make computation possible, let’s go back to the image with the red circles and blue squares. First, we need vectors to represent the SHAPE and COLOR variables. Then we also need vectors for the values that can be assigned to variables: CRITICAL, SQUARE, BLUE and RED.
The vectors must be distinct. This difference can be quantified by a property called orthogonality, that is, at right angles. In 3D space, there are three vectors that are orthogonal to each other: a vector x direction, different in y, and a third in z. In 10,000-dimensional space there are 10,000 such orthogonal vectors.
