The future of AI compute is photonics… or is it?
Exploring the potential of photonics to improve efficiency in the future of compute
Riding the wave with analog computing
The concept of analog computing dates back to the early 20th century when analog computers were widely used for scientific and industrial applications such as predicting tides and simulating flight dynamics. None other than Claude Shannon, the father of information theory, published on the use of analog computing in solving differential equations back in 1941 for use in physics, engineering, and economics.
Tide-predicting computer; British Association for the Advancement of Science (per Lord Kelvin)
Since then, the definition of analog computing has broadened somewhat, but in basic terms it encompasses one or more of the following:
1. Processing and memory combined in non von Neumann architectures
2. Inherently imprecise & noisy rather than binary and high precision
3. Operates on continuous signals rather than sampling and digitising information
The principle of combining memory elements and processing units has been discussed previously in articles on neuromorphic computing and in-memory compute, the latter also touching on the use of analog techniques to further reduce energy consumption albeit with some notable drawbacks.
This short article explores the third aspect of analog computing, namely the use of continuous signals rather than the discrete binary digits used in modern computing, and in particular wave-based analog computing.
In such an approach, data is encoded in the properties of electromagnetic waves, such as the amplitude, phase, frequency, or polarisation. Mathematical operations can then be carried out by manipulating these wave properties as signals propagate through the system.
By eliminating the inefficiencies of analog/digital conversion, wave computers can operate ‘at the speed of light’, performing operations with extremely low latency and enabling massively parallel computation. Optalysys, for instance, uses this approach to enable more efficient computation of FFTs for fully homomorphic encryption (FHE) in privacy and security applications.
Wave-based analog computers could pave the way for ultra-fast, on-the-fly power-efficient data processing encompassing a range of operations including matrix-vector multiplication, discrete Fourier transforms, filtering, and solving complex matrix equations.
Wave computing can also solve differential equations much faster than their digital counterparts by directly modelling the continuous changes described by the equations. In addition to the fields of physics, engineering and economics mentioned earlier, the techniques can equally be applied in solving equations relevant to biology, fluid dynamics, weather prediction, quantum chemistry, plasma physics, and many other scientific fields.
Ordinarily, such an approach would require bulky optical components, but with the development of metamaterials it has become possible to engineer artificial structures that are carefully designed with respect to their shape, geometry, size, orientation and arrangement to control how electromagnetic waves interact with them thereby enabling subwavelength building blocks and the realisation of chip-level all-optical systems.
Source: https://www.photonics.com/Articles/Optical_Metamaterials_Use_III-V_Semiconductors_as/a61869
On the flip-side, such reliance on metamaterial fabrication limits a given design to specific, fixed functions determined by the physical structure, and precludes the wave computer being easily reconfigured or reprogrammed for different tasks – in this regard, it is more analogous to an ASIC than an xPU.
Another drawback of using metamaterials is their weak non-linearity, hence application has typically been confined to simple linear operations such as image edge detection or matrix multiplications rather than more complex computing tasks requiring strong non-linearities such as in realtime signal processing for use in telecoms and radar systems, or the non-linear activation functions used in ANNs (e.g., ReLU, Sigmoid).
Analog wave-based computing remains an intriguing prospect. Its drawbacks in size and wider applicability has so far limited it to a few specific compute tasks, and hence interest has been more academic than commercial to-date. But in a wider landscape of heterogeneous compute, with different computing paradigms being employed depending on the task, wave-based computing still deserves to be in the mix, and hence an interesting space for startups to explore.
