New Artificial Neuron Device Could Greatly Reduce Energy Requirements for Neural Computations - Unite.AI

Researchers at the University of San Diego have developed a new artificial neuron device that can run neural computations with 100 to 1000 times less energy and area than the current CMOS-based hardware.

The recent report was published in a paper on March 18 in Nature Nanotechnology.

In order to generate the input for one of the connected layers of artificial neurons in neural networks, a mathematical calculation called a non-linear activation function must be applied. However, this application requires a lot of computer power and circuitry due to the need to transfer data between the memory and external processor.

The researchers at UC San Diego developed a new nano-meter sized device that can carry this activation function out much more efficiently.

Duygu Kuzum is a professor of electrical and computer engineering at the UC San Diego Jacobs School of Engineering.

“Neural network computation in hardware gets increasingly inefficient as the neural network models get larger and more complex,” said Kuzum. “We developed a single nanoscale artificial neuron device that implements these computations in hardware in a very area- and energy-efficient way.”

The study was led by both Kuzum and Ph.D student Shangheon Oh, and they collaborated with UC San Diego physics professor Ivan Schuller, who leads a DOE Energy Frontier Research Center. The center is involved with developing hardware implementations of energy-efficient artificial neural networks.

The New Device

The newly developed device relies on a rectified linear unit, which is one of the most common activation functions used in neural network training. It requires hardware that can undergo gradual changes in resistance, which is what the engineers aimed towards. The device can gradually switch from an insulating to conducting state with a small of amount of heat.

Termed a Mott transition, this switch occurs in an extremely thin layer of vanadium dioxide, and above this layer is a nanowire heater made of titanium and gold. The vanadium dioxide layer slowly heats up when current flows through the nanowire, and this causes a slow and controlled switch from insulating to conducting.

Oh is the study’s first author.

“This device architecture is very interesting and innovative,” said Oh. “In this case, we flow current through a nanowire on top of the material to heat it and induce a very gradual resistance change.”

Implementing the Device

For implementation, the team fabricated an array of activation devices and a synaptic device array, followed by the integration of the two on a custom printed circuit board. They were then connected together, resulting in a hardware version of a neural network.

The network was used to process an image through edge detection, identifying the outlines and edges of objects in the image. The integrated hardware system demonstrated its ability to perform convolution operations that are important for various types of deep neural networks.

“Right now, this is a proof of concept,” Kuzum said. “It's a tiny system in which we only stacked one synapse layer with one activation layer. By stacking more of these together, you could make a more complex system for different applications.”