right now, a GPU is spending most of its energy not on math, but on moving numbers around. Fetching weights from memory, shuttling them to a compute unit, doing a multiplication, and shuttling the result back. Do that billions of times a second, for months, across a training run, and you start to understand why AI’s electricity bill has become a boardroom problem rather than an engineering footnote.
The numbers back this up. Gartner expects global data center power demand to climb from 104 gigawatts in 2025 to 132 gigawatts in 2026, and to reach roughly 290 gigawatts by 2030, with generative AI workloads named as the primary driver. AI-optimized servers alone are projected to account for nearly a third of data center power consumption this year.
This is the backdrop for a quiet resurgence of an old idea: analog computing. Actual analog circuits, using continuous voltages and currents instead of 1s and 0s, to do the matrix multiplications that dominate neural network inference. The pitch is seductive: instead of computing about physics digitally, why not let physics compute directly?
It’s not a new idea, of course, analog computers predate digital ones. It was abandoned decades ago for good reasons. And this article is about both halves of that story: why the pitch is truly compelling, and why what killed analog computing the first time hasn’t actually gone away. I’ll walk through the mechanism, then run some small, honest simulations that show exactly where it breaks, and how researchers are starting to patch it.
How analog in-memory computing actually works
In a conventional chip, memory and compute are separate. Every time you multiply a weight by an input, that weight has to travel from a memory array, across a bus, into a compute unit, and the result has to travel back. Researchers estimate this movement can cost anywhere from 3 to 10,000 times more energy than the arithmetic itself, that’s the so-called von Neumann bottleneck.
Analog in-memory computing (AIMC) collapses that trip. Weights are stored as physical conductance values in a grid of memory cells, often built from phase-change memory or resistive RAM. Apply an input as a voltage across that grid, and two laws of physics do the rest: Ohm’s law (current = voltage × conductance) computes each multiplication, and Kirchhoff’s current law (currents sum when wires meet) computes the accumulation. A full matrix-vector multiply, the operation neural networks spend most of their time on, happens in a single physical step, no bus required.
There’s a catch hiding in that sentence, though: the multiplication itself is essentially free, but the input and output still have to cross the boundary between the analog and digital worlds: voltages in, currents out, both needing conversion. In practice, the ADCs and DACs doing that conversion are often where the real energy and area budget goes, not the crossbar array itself. It’s a bit of an irony at the heart of the field: the physics does the math for free, and then a meaningful share of the chip is spent paying to talk to it.
This isn’t hypothetical. IBM Research’s HERMES project chip is a working example, a phase-change-memory-based analog in-memory chip that’s already been used to demonstrate near-software accuracy on real workloads. An earlier IBM PCM-based prototype, reported in 2023, packed 35 million phase-change memory cells and held up to 17 million parameters directly on-chip, at a fraction of the energy of a digital accelerator. Startups like EnCharge AI (spun out of Princeton, working with TSMC on manufacturing) and Mythic have raised real money betting the same physics can be productized for edge inference.
Simulating it yourself
You don’t need exotic hardware to see the mechanism at work. Here’s a minimal simulation: a “perfect” digital matrix multiply, compared against the same operation with realistic analog imperfections layered in, device-to-device programming noise (weights don’t get written to the exact conductance you asked for), read noise (thermal/shot noise every time you use the array), and finite ADC precision (the analog result still has to be converted back to a digital number eventually).
import numpy as np
rng = np.random.default_rng(42)
n_in, n_out = 64, 32
W = rng.uniform(-1, 1, size=(n_in, n_out)) # "ideal" weights
x = rng.uniform(-1, 1, size=n_in) # input activations
digital_result = x @ W # ground truth
def analog_matmul(x, W, programming_noise_std=0.0, read_noise_std=0.0, adc_bits=None):
# weights are physically stored as conductances -> noise is baked in
# until the cell is reprogrammed
G = W + rng.normal(0, programming_noise_std, size=W.shape)
# Ohm's law + Kirchhoff's current law, in one step
out = x @ G
# thermal/read noise added at measurement time
out = out + rng.normal(0, read_noise_std, size=out.shape)
# the analog result still needs to be digitized
if adc_bits is not None:
lo, hi = out.min(), out.max()
step = (hi - lo) / (2 ** adc_bits)
out = np.round((out - lo) / step) * step + lo
return out
Running this with increasing amounts of realism produces a clean, escalating error curve:
Going from a perfect multiply to one with both noise sources and a cheap 4-bit ADC introduces over 8% relative error, on a single layer, before that error has any chance to compound across a deep network. Which raises the real question: how much does that actually matter for a trained model’s accuracy?
Analog computing has always had a noise problem
This is exactly why analog computing lost to digital in the first place, decades before AI made it relevant again. Continuous physical signals are inherently harder to pin down than binary states. A digital circuit only has to distinguish a 0 from a 1; an analog circuit has to preserve a precise magnitude against thermal noise, device variation, and drift over time. It’s a much harder engineering problem, and it doesn’t go away just because the application changed from solving differential equations to running neural networks. Recent work cataloguing these effects lists programming noise, read noise, temporal conductance drift, limited bit precision, and analog-to-digital conversion loss as the standing open problems in the field. Drift is a particularly strange one to sit with: unlike a digital bit, an analog weight is a physical material property, and that material relaxes over time, so the “same” weight can quietly read differently a week after it was written, which is why analog chips typically need periodic recalibration or reprogramming just to stay accurate, an overhead digital memory doesn’t pay.
To see whether this actually matters, I trained a small neural network the normal way, with full digital precision, no noise anywhere, on the classic handwritten digit classification task, then ran inference through a simulated analog layer at increasing noise levels.
def forward(params, X, layer2_noise_std=0.0):
h = np.maximum(0, X @ params["W1"] + params["b1"]) # digital hidden layer
W2 = params["W2"]
if layer2_noise_std > 0:
# this line is the analog crossbar: weights corrupted by device noise
W2 = W2 + rng.normal(0, layer2_noise_std, size=W2.shape)
logits = h @ W2 + params["b2"]
return softmax(logits), h
The network was trained with zero knowledge that it would ever see noise. Here’s what happened when I evaluated it through progressively noisier simulated hardware:
The network is remarkably tolerant of small amounts of noise, as accuracy barely moves below noise std ≈ 0.1. But past that point, it doesn’t degrade gracefully, it falls off a cliff: 83% at 0.2, 64% at 0.4, and by 0.8 it’s approaching random guessing. This is the real shape of “analog AI has a noise problem”, not a gentle accuracy tax, but a threshold the network was never prepared to cross.
Training the network for the hardware it’ll run on
The current research response to this isn’t (just) about building quieter hardware, it’s about not training the network to expect a clean signal in the first place. The technique goes by a few names depending on who’s publishing it, IBM calls a version of it hardware-aware training, and a recent line of research on what it terms “analog foundation models” calls it noise-injection training, but the idea is the same: simulate the target hardware’s noise during training, not just at deployment, so gradient descent is forced to find weights that are robust to it.
I reran the same experiment, but this time trained a second copy of the network with noise injected into every forward pass during training:
def train(params, X, Y, epochs=300, lr=0.1, train_noise_std=0.0):
for epoch in range(epochs):
# train_noise_std > 0 means the network never sees a clean signal —
# it has to learn weights that tolerate the noise from day one
probs, h = forward(params, X, layer2_noise_std=train_noise_std)
# ...gradient descent as normal from here
The result is the most interesting chart in this piece:
At zero noise, the two models are statistically identical: noise-aware training costs essentially nothing when the hardware happens to be clean. But as noise increases, the gap opens fast: at std=0.6, the normally-trained model has collapsed to 39% while the noise-aware model is still holding 61%. Neither model is immune to noise, as always, there’s no free lunch, and accuracy still declines, but one of them degrades like a system that was designed for this, and the other degrades like a system that was never told what it was going to run on.
This is just a toy example, a two-layer network on 8×8 digit images, not a real production model, but it’s the same mechanism researchers are now applying to LLMs, with the added complication that large language models have far more layers for noise to compound across, and far less tolerance for the kind of accuracy cliff shown above.
Where this actually stands
As a summary, it’s worth being honest about the state of the field. A few things seem well established:
It looks real for inference, not yet for training. Nearly every serious source on this draws the same line: analog hardware is viable for running an already-trained model, especially at the edge, but the precision demands of backpropagation make training on analog substrates a much harder unsolved problem.
The commercial track record is mixed. Mythic, one of the earliest analog AI chip startups, has spent years navigating a market where digital low-power chips keep improving too, even as it raised a $125 million round in late 2025. EnCharge AI, founded in 2022, has moved faster on manufacturing partnerships, working with TSMC and claiming roughly 20x lower energy use than comparable digital chips, though that claim is still largely pre-commercial. IBM remains mostly in research mode, recently publishing on mapping mixture-of-experts LLM layers onto 3D analog memory architectures.
It’s one bet among several, not the whole story. Photonic computing (using light instead of voltage, pursued by companies like Lightmatter and Celestial AI) is a related but distinct wager, and today it’s aimed more at solving chip-to-chip interconnect bottlenecks than at replacing the compute itself. Neuromorphic chips like Intel’s Loihi line take yet another approach, borrowing the brain’s event-driven, spike-based signaling rather than dense analog matrix math. All three get lumped together in press coverage as “brain-inspired computing,” but they’re solving different parts of the problem.
None of this fixes AI’s power crisis by itself, and it shouldn’t be sold that way. But it’s a genuine architectural rethink at a moment when digital efficiency gains are running out of easy wins — and unlike most hardware roadmaps, you can actually verify the central claim, and the central problem, from your laptop in about thirty lines of numpy.
Sources and further reading
- Gartner, Data Center Electricity Consumption to Grow 26% in 2026
- IBM Research, Analog in-memory computing could power tomorrow’s AI models
- IBM Research, The AIU family of chips
- IBM Research, NorthPole: an architecture for neural network inference with a 12nm chip (digital, not analog — included for the disambiguation)
- Büchel et al., Kernel Approximation using Analog In-Memory Computing — HERMES project chip, IBM’s PCM-based analog chip
- IEEE Spectrum, The Femtojoule Promise of Analog AI
- IEEE Spectrum, EnCharge’s Analog AI Chip Promises Low-Power and Precision
- TechCrunch, EnCharge raises $100M+ to accelerate AI using analog chips
- “Analog Foundation Models Tackle AI Hardware Noise for LLMs” — neurotechnus.com