AI faces an energy challenge. Will chip design overcome it?
The AI boom's impact on energy consumption has been an increasing focal point of attention as adoption of AI grows across the globe. Tech companies have significantly increased AI-related investments, and while average data centres have comparatively modest power demands, increasingly popular 'hyperscale' facilities can consume as much power as an electrified steel mill.
Goldman Sachs estimates that by 2030, data centres will use 3-4% of global power, and, in some regions, that figure will be much higher. In Denmark, data centre power demand is expected to increase sixfold, reaching nearly 15% of the country’s electricity use. In Ireland, 21% of metered electricity already goes to data centres.
This surging demand poses both reputational and operational challenges for tech companies, with Gartner predicting that power shortages will restrict 41% of AI data centres by 2027.
'Micro' AI models run on smartphones and laptops are one response to this challenge, already being developed by frontier leaders in AI model development, but chip design can also play a role. Recent years have seen marked improvements in the energy efficiency of AI hardware. The International Energy Agency publishes an index of the energy intensity of AI-related computer chips and reports that this metric has decreased from 100 in 2008 to less than one in 2023.
Will chip efficiency solve AI’s energy problem? Today's GPUs are modified from designs for video game graphics and experts see potential in AI-specific chips. They also suggest further ways in which AI hardware can be made more efficient.
Analog transformation
In common parlance, ‘analogue’ is synonymous with ‘old-fashioned’. But analogue, or ‘in-memory’, computing is a cutting-edge area of chip development that improves energy efficiency by reducing the movement of data and electrons. This is achieved by processing data where it is stored, rather than shuttling it back and forth between the memory and processor, as is the case with today’s chips.
UMass Amherst professor Qiangfei Xia is a leading figure in analogue computing. His team recently announced that their device, called a memristor, can perform complex scientific computing tasks. “Our research in the past decade has made analog memristor a viable technology,” he explains. “It is time to move such a great technology into the semiconductor industry to benefit the broad AI hardware community.”
Could photonic computing light the way?
Traditionally, data has been processed, communicated, and stored using electrical currents. ‘Photonic’ or ‘optical’ computing is a promising alternative approach that replaces electrons with light waves.
German technical-scientific association VDE highlights that photonic integrated circuits have higher energy efficiency, in part, because they generate less heat. “Photonic technologies could play a key role in reducing the energy requirements of data centres while meeting the growing demands for speed and computing power,” explains Dr. Matthias Wirth, Project Manager, Innovation at VDE.
Researchers at MIT recently published details about a ground-breaking photonic chip that can perform AI tasks that had previously proven difficult for light-based systems. Crucially, this chip was manufactured using commercial foundry processes.
Mimicking the human brain
The human brain is the inspiration for yet another innovative approach known as ‘neuromorphic’ computing.
Sander Bohté from Dutch research institute CWI, has been working in this field since the late 1990s, developing a technology called ‘spiking neural networks’. These more closely resemble the brain by transmitting information through pulses rather than a continuous signal.
The challenge with this approach is its mathematical complexity, but Bohté and his team have developed algorithms that, when paired with a special type of chip, can deliver significant energy efficiency gains. “When our algorithm is run on this special chip, a factor of twenty in energy reduction is gained,” he explains.
Developing end-to-end energy efficiency
While we are seeing an extraordinary spike in private-sector hardware investment, Oak Ridge National Laboratory has decades of experience delivering high-performance supercomputers under strict power constraints. In a September article, the organisation argued that this knowledge can help private companies during the current boom.
The Lab advocates for a holistic approach to energy efficiency that tackles everything from the applications to the facilities to the hardware. “You need efficiency gains in all three of those areas to attack the problem,” explains OLCF program director Ashley Barker. “There are some possible technologies out there that might give us some jumps, but the biggest thing that will help us is a more integrated, holistic approach,” added Scott Atchley, CTO of the National Center for Computational Sciences at ORNL. One of the Lab’s focus areas is the use of telemetry data to build up energy profiles of each computing job, which can inform better operational decision-making.
Hardware improvements have a history of reducing the energy intensity of computing, and chips have become steadily more efficient over time. There is therefore every reason to be hopeful that continued innovation could help to solve AI’s current energy challenge.