Analog computers could bring massive efficiency gains to machine learning

In The Next Generation of Deep Learning Hardware: Analog Computing *Sci-Hub mirror), a trio of IBM researchers discuss how new materials could allow them to build analog computers that vastly improved the energy/computing efficiency in training machine-learning networks.

Training models is incredibly energy-intensive, with a single model imposing the same carbon footprint as the manufacture and lifetime use of five automobiles.

The idea is to perform matrix multiplications by layering "physical array with the same number of rows and columns as the abstract mathematical object" atop each other such that "the intersection of each row and column there will be an element with conductance G that represents the strength of connection between that row and column (i.e.,the weight)."

Though these would be less efficient for general-purpose computing tasks, they would act as powerful hardware accelerators for one of the most compute-intensive parts of the machine learning process.

Electrochemical devices are a newcomer in the field of contenders for an analog array element for deep learning.The device idea, however, has been around for a long time and is related to the basic principle of a battery [52]. Compared to the previously discussed switches, which only required two terminals, this switch required three terminals. The device structure, shown in Fig. 11, is as tack of an insulator that forms the channel between two contacts source and drain, an electrolyte, and a top electrode (reference electrode). Proper bias between the reference electrode and the channel contacts will drive a chemical reaction at the host/electrolyte interface in which positive ions in the electrolyte react with the host, effectively doping the host material.