As deep neural networks drive advancements in machine learning, energy consumption and throughput are becoming key limitations of traditional complementary metal–oxide–semiconductor (CMOS) electronics. This has spurred the exploration of new hardware architectures optimized for artificial intelligence, including electronic systolic arrays, memristor crossbar arrays, and optical accelerators.
Optical systems, capable of performing linear matrix operations with high speed and efficiency, have led to recent developments in low-latency matrix accelerators and optoelectronic image classifiers. However, achieving coherent, ultralow-latency optical processing for deep neural networks remains a significant challenge.
Scientists from MIT and other institutions have developed a new photonic chip that addresses the limitations of traditional hardware in deep neural network processing. They demonstrated a fully integrated photonic processor capable of performing all key computations of a deep neural network optically on the chip.
The optical device completed the necessary computations for a machine-learning classification task in under half a nanosecond, achieving over 92 percent accuracy—comparable to traditional hardware performance. The chip, consisting of interconnected modules that form an optical neural network, is fabricated using commercial foundry processes, which could enable the technology’s scaling and integration into electronics.
Our brains use quantum computation
In the long term, the photonic processor could enable faster and more energy-efficient deep learning for computationally intensive applications such as lidar, scientific research in astronomy and particle physics, and high-speed telecommunications.
According to the researchers, the speed of obtaining results is as vital as the model’s performance in many cases. With an end-to-end system that can run a neural network in optics at a nanosecond timescale, new possibilities for applications and algorithms can now be explored at a higher level.
Saumil Bandyopadhyay ’17, MEng ’18, PhD ’23, a visiting scientist in the Quantum Photonics and AI Group within the Research Laboratory of Electronics (RLE) and a postdoc at NTT Research, Inc. said, “Nonlinearity in optics is quite challenging because photons don’t interact with each other very easily. That makes it very power consuming to trigger optical nonlinearities, so it becomes challenging to build a system that can do it in a scalable way.”
Deep-learning model to’ reverse-engineer’ new composite materials
The researchers overcame the challenge of integrating nonlinear operations by designing devices called nonlinear optical function units (NOFUs), which combine electronics and optics to perform nonlinear operations directly on the chip. Using these NOFUs, they built an optical deep neural network on a photonic chip, incorporating three layers of devices that handle linear and nonlinear operations.
Initially, the system encodes the parameters of a deep neural network as light. Next, a set of programmable beamsplitters, introduced in a 2017 paper, carries out matrix multiplication on these inputs. The data are then directed to programmable nonlinear optical function units (NOFUs), which perform nonlinear operations by redirecting a small portion of the light to photodiodes that convert the optical signals into electric current. This method removes the need for an external amplifier and is highly energy-efficient, using minimal power.
Bandyopadhyay says, “We stay in the optical domain until the end when we want to read out the answer. This enables us to achieve ultra-low latency.”
An on-chip photon-counting device to advance quantum technology
Achieving such low latency allowed the researchers to efficiently train a deep neural network directly on the chip, a process known as in situ training, which typically requires a significant amount of energy in digital hardware. “This is particularly beneficial for systems that perform in-domain processing of optical signals, such as navigation or telecommunications, and in systems that require real-time learning.”
The photonic system reached over 96 percent accuracy during training tests and more than 92 percent accuracy during inference, similar to traditional hardware’s performance. Furthermore, the chip completes key computations in under half a nanosecond.
“This research shows that computing — essentially the process of mapping inputs to outputs — can be implemented on new architectures based on linear and nonlinear physics, leading to a fundamentally different scaling law between the computational effort and the required resources,” the researchers stated.
The entire circuit was created using the same infrastructure and foundry processes in producing CMOS computer chips. This approach could allow the chip to be mass-produced using reliable techniques that minimize fabrication errors.
Bandyopadhyay notes that a key focus for future work will be scaling up the device and integrating it with real-world electronics, such as cameras or telecommunications systems. Additionally, the researchers aim to explore algorithms that can use optical technology to accelerate training and improve energy efficiency.
Journal Reference:
- Bandyopadhyay, S., Sludds, A., Krastanov, S. et al. Single-chip photonic deep neural network with forward-only training. Nat. Photon. 18, 1335–1343 (2024). DOI: 10.1038/s41566-024-01567-z
