For now, after a quick look, it seems that this spans quite a broad range…:
The central idea—representing trained parameters in a physical substrate and allowing many operations to occur in parallel—has several real precedents. The closest areas seem to be analogue/compute-in-memory hardware, spatial FPGA or ASIC dataflow, neuromorphic computing, and, somewhat further away, physical neural networks.
What appears most established at present is not a complete one-to-one replacement of every software neuron with an independent circuit. The more common route is to move a particularly expensive part of the network—usually a matrix-vector multiplication, a group of layers, or a fixed inference pipeline—into a specialized physical array, while retaining digital control, communication, activation functions, calibration and unsupported operations around it.
So I think the useful first question is not simply whether the idea is possible. It is:
Which part of the model should become physical, how often must it be reprogrammed, and which operations must remain flexible?
Those choices lead to rather different designs.
| Design axis |
Some possible branches |
| Physical unit |
Individual weight, neuron, matrix tile, layer, sensor front-end, or an entire physical dynamical system |
| Model lifecycle |
Fabrication-fixed, occasionally programmed, frequently replaced, digitally adapted, or trained on-chip |
| Workload |
Dense fixed matrix multiplication, small low-latency network, dynamic attention/KV state, sparse event-driven processing, or continuous physical dynamics |
A relatively conservative starting route would be:
- choose a small, mostly fixed inference model;
- identify one dense linear block or small MLP/CNN that dominates its work;
- implement that block as a spatial FPGA/ASIC design or an analogue in-memory tile;
- keep programming, readback, calibration and unsupported operations digital;
- compare it with a digital implementation using the same numerical precision.
That route does not complete the whole roadmap, but it exposes the important boundaries without requiring all of them to be solved at once.
Small, mostly fixed dense inference
-> FPGA/ASIC spatial dataflow or analogue in-memory tile
Processing close to a sensor
-> in-sensor or near-sensor partial replacement
Frequent model replacement
-> programming accuracy, verification, remapping and recalibration matter
One base model with several tasks
-> a fixed physical base with smaller digital adapters may be sufficient
Full Transformer or LLM
-> separate static linear weights from attention, KV state,
nonlinear/global operations, communication and host control
Sparse event-driven workload
-> neuromorphic or spiking architecture
Use the physical dynamics themselves
-> physical neural network
Where similar ideas already exist
Early electrically programmable neural-network hardware
There is a fairly direct historical precedent in Intel’s 80170 ETANN, announced in 1989. ETANN meant “Electrically Trainable Analog Neural Network”; the chip contained 64 analogue neurons and 10,240 floating-gate synapses. The Computer History Museum overview also points to contemporary papers and development-system material.
That example is useful because it shows that electrically stored weights and physical analogue neurons are not new in principle. It also shows that the chip itself was only one component of a larger system involving training, testing, I/O and development tools.
It therefore supports the general direction, but it is not evidence that a modern large model can be mapped one neuron at a time without substantial surrounding infrastructure.
Analogue in-memory computing
The closest modern field is probably analogue in-memory computing or compute-in-memory.
A common implementation stores weights as conductance values in a memory array. Input values are converted into voltages or pulses, and the resulting currents perform many multiply-accumulate operations in parallel through the physical behaviour of the array.
This is slightly different from assigning a separate DAC to every weight. In many designs:
- the weight is represented by a memory-device state;
- DACs or pulse generators encode the inputs;
- column currents represent accumulated results;
- ADCs and digital scaling recover usable output values.
IBM’s 64-core mixed-signal in-memory-compute chip is a useful system-level example. It integrates 64 analogue compute cores, an on-chip communication network, digital activation functions and other processing needed by convolutional and recurrent layers.
That combination is important: the analogue array is not treated as a complete computer by itself. It is one efficient component inside a heterogeneous system.
The open-source IBM Analog Hardware Acceleration Kit can be used to explore how neural-network layers behave under analogue-device noise, finite converter precision, drift and other hardware effects. AIHWKit-Lightning extends this direction toward larger networks and hardware-aware training.
CrossSim is another useful simulator for studying how resistive-crossbar properties affect algorithmic accuracy. It can model matters such as array size, parasitic resistance, weight errors, ADC precision and read noise. Its scope should be kept clear: it is primarily an accuracy and co-design simulator, not proof of the speed, area or energy of a fabricated complete system.
FPGA and ASIC spatial dataflow
The same basic idea can be explored without analogue devices by translating a trained network into a dedicated FPGA or ASIC data path.
hls4ml converts supported neural networks into high-level-synthesis designs. Its concepts documentation provides a particularly clear illustration of the main trade-off:
- a low reuse factor executes more multiplications in parallel and reduces latency;
- a high reuse factor shares hardware across operations and reduces resource consumption;
- fully parallel I/O can work well for MLPs and small CNNs;
- the same approach can consume too many resources or fail synthesis for larger networks.
This seems closely related to the proposed “all neurons in parallel” idea. It is a valid implementation point, but not a free one. Physical area, routing, buffers and available multipliers replace instruction count as the limiting resource.
The FINN project explores a related route for quantized neural networks, generating customized dataflow accelerators rather than treating the network as ordinary GPU software.
Specialized digital accelerators
The first TPU is a useful example of a less literal but successful form of replacement. It did not reproduce every neuron as an individual circuit. Instead, it specialized the dominant matrix-multiplication workload while retaining a host and a normal software interface.
Google’s TPU performance analysis is therefore relevant as a boundary-design precedent:
It may not be necessary to replace the entire computer to remove a large fraction of the inefficient work.
A narrower replacement can also preserve more programmability and make deployment substantially easier.
Neuromorphic computing
Neuromorphic systems such as Intel’s Loihi 2 also place many neuron-like state machines and local memories in hardware. Intel describes Loihi-based systems as using asynchronous, event-based spiking networks, integrated memory and computation, and sparse communication; Hala Point is a large research system built from Loihi 2 processors.
This is related, but it is not simply a conventional dense neural network transferred into transistors.
The model generally changes as well:
- communication occurs through sparse events or spikes;
- neurons may have persistent internal state;
- computation can be asynchronous;
- inactive parts of the network need not continuously operate.
For dense Transformer-style matrix multiplication, this is therefore a different branch rather than an interchangeable implementation technology.
Physical neural networks
A still broader branch is to stop requiring the physical hardware to imitate a conventional artificial neuron exactly.
The work on deep physical neural networks treats controllable optical, mechanical and electronic systems as trainable physical layers. Physics-aware training compensates for the difference between a software model and the real system.
This opens a different possibility:
Instead of asking how to reproduce an abstract neuron in hardware, ask which useful transformation the physical system already performs efficiently and train around that transformation.
This is less directly aligned with a conventional interchangeable neural-network chip, but it may be relevant if “Zero Compute” primarily means allowing the substrate’s own dynamics to perform the transformation.
What maps well, and what changes the answer
Dense and mostly static linear operations
Physical arrays are most naturally matched to operations where:
- the same weight matrix is reused many times;
- the matrix is large enough to keep the array occupied;
- the computation is dominated by multiply-accumulate operations;
- inputs and outputs can be converted efficiently;
- the required precision is moderate;
- weights change much less often than activations.
This includes many fully connected layers and convolutional blocks.
The benefit is not only that many products occur simultaneously. Weight data also does not need to be repeatedly fetched from a separate external memory.
Small or poorly utilized operations
The answer changes when an operation cannot fill the physical array efficiently.
Examples include:
- small matrices;
- depthwise or highly grouped operations;
- irregular tensor shapes;
- sparse work without suitable sparse routing;
- operators whose execution is dominated by indexing or control;
- layers that require much higher numerical precision than the main matrix core.
A large array can be theoretically parallel while spending much of its area idle. In that case, a smaller digital unit or a reused multiplier bank may be more efficient.
Nonlinear and global operations
Matrix multiplication is only one part of a network.
Depending on the architecture, the system may also need:
- activation functions;
- normalization;
- pooling;
- reductions;
- exponentiation or division;
- Softmax;
- residual additions;
- routing and reshaping;
- sampling and output control.
Some of these can be approximated physically or implemented with lookup tables. Others may be cheaper and more flexible in digital logic.
Accelerating the matrix core can therefore expose a new bottleneck in the surrounding operations.
Layer dependencies remain
Neurons within a layer may operate simultaneously, but the next layer still depends on the previous layer’s output.
Several layers can be pipelined so that different inputs occupy different stages at the same time. This improves throughput, but does not make an arbitrarily deep dependency chain disappear.
There is consequently another design choice:
Fully spatial design
-> separate hardware for many or all layers
-> low latency and high throughput
-> high area and routing cost
Layer-serial design
-> reuse a physical block across several layers
-> lower area and easier programming
-> additional latency and data movement
Intermediate tiled design
-> map groups of layers to physical tiles
-> trade area, communication and utilization
Interconnect and fan-out
Once arithmetic becomes cheap, wires can become the expensive part.
A large network may require:
- distributing one activation to many destinations;
- summing partial outputs from multiple tiles;
- buffering signals between stages;
- crossing clock, voltage or analogue/digital boundaries;
- maintaining signal integrity over a large physical area.
This is one reason current systems tend to use tiles and communication networks rather than one unrestricted physically connected graph.
The physical layout of the model can become as important as the number of neurons.
Processing only the first or most expensive boundary
Another possible route is to place computation close to the sensor and avoid digitizing or transmitting all raw data.
For example, a vision device could perform an early convolution, filter or compression stage near the sensor, then send a smaller representation to an ordinary processor.
This may capture much of the data-movement benefit without requiring the entire model to become physical.
Programming, observability, calibration and model replacement
The possibility of overwriting the non-volatile weights is important, but “electrically programmable” can cover several different levels of control.
A complete model-loading path may look more like this:
Logical trained weights
-> quantization and encoding
-> tile placement and routing
-> physical programming
-> readback or verification
-> scale and offset calibration
-> fault remapping
-> end-to-end accuracy validation
Programming accuracy
In a digital memory, writing a value generally produces that exact discrete bit pattern.
In an analogue device, a requested weight may become a conductance, voltage, charge, optical phase or another continuous physical state. The final state can depend on:
- device-to-device variation;
- previous programming history;
- nonlinear response;
- temperature;
- write-pulse shape;
- saturation;
- stochastic switching.
Some technologies therefore use a write–read–adjust loop until the measured state is sufficiently close to the target.
This does not mean every possible physical substrate must require slow closed-loop programming. For example, an open-loop ECRAM array demonstrated accurate analogue programming without feedback adjustments for its tested array and tasks.
The useful distinction is therefore not “analogue devices are uncontrollable”, but:
What programming accuracy is available for this particular substrate, at this scale, after repeated writes and over the required lifetime?
Observability
A state cannot be calibrated unless it can be measured.
Useful questions include:
- Can each weight be read independently?
- Is only a tile-level input/output response observable?
- Does reading disturb the stored state?
- What conversion precision is available?
- Can the system distinguish a programming error from a routing or converter error?
- Is there a known reference input for calibration?
The measurement path itself requires circuits, time and energy. Those costs belong in the system design even if they are not on the fast inference path.
Stability
A successfully programmed state may still change with:
- time;
- temperature;
- repeated reads;
- supply variation;
- neighbouring activity;
- material drift;
- device ageing.
The required solution depends strongly on the application.
Stable environment, fixed model
-> initial calibration may be sufficient
Slow predictable drift
-> occasional reference measurements and recalibration
Frequent environmental changes
-> online compensation or feedback
Frequent model replacement
-> programming and recalibration become part of deployment latency
On-chip training
-> update linearity, symmetry, endurance and write energy become central
Hardware-aware training is one way to make models more tolerant of expected device and circuit non-idealities. It does not remove the physical errors; it trains the model to operate despite a modeled range of them.
Cross-coupling
Individual controls may not remain independent.
Changing one physical parameter can affect another through:
- electrical parasitics;
- voltage drops along an array;
- capacitive coupling;
- optical thermal cross-talk;
- shared supply or reference lines;
- physical device interactions.
If this is substantial, programming one weight at a time may invalidate an earlier setting. The control problem then belongs to the complete array, not to an isolated weight.
Faults and remapping
At a sufficiently large scale, some devices or routes may be outside tolerance.
A programmable system may therefore need:
- spare rows or columns;
- disabled devices;
- duplicated weights;
- alternative routes;
- tile remapping;
- fault-aware training or placement.
This is another reason why a compiler and runtime can be as important as the neuron circuit itself.
Model replacement
A new release can still be loaded into non-volatile hardware, but the relevant deployment metric is not only the raw write time.
It may be useful to measure:
weight-encoding time
physical programming time
verification time
placement and routing time
calibration time
energy per model update
accuracy immediately after programming
accuracy after temperature and time changes
number of supported rewrite cycles
If the base model changes rarely but tasks change frequently, another architecture is possible: keep the large base weights in the physical substrate and place smaller task-specific adapters in programmable digital memory.
That would preserve the main data-locality advantage without reprogramming every physical weight for every update.
System architecture and software stack
A useful separation is:
Compute plane
matrix operations
activation processing
local routing
inference data flow
Control plane
model loading
programming and verification
calibration
fault detection
remapping
scheduling
monitoring
A system can have a very efficient compute plane while still requiring a conventional processor for the control plane.
Analogue/digital boundaries
A mixed-signal implementation may contain:
- input DACs or pulse generators;
- analogue memory arrays;
- current or charge accumulation;
- ADCs;
- digital scaling and accumulation;
- digital activation units;
- an inter-tile network;
- local controllers;
- a host processor.
Converter precision has a direct cost in area and energy. Increasing the internal weight precision is not enough if the inputs, accumulators or outputs cannot preserve it.
This is why ADC/DAC configuration, bit slicing and accumulation strategy are usually first-class architecture choices rather than minor implementation details.
Heterogeneous architectures
The 64-core PCM chip combines analogue matrix operations with an on-chip network and digital functions.
The more recent EAGLE architecture likewise combines analogue compute-in-memory units with programmable RISC-V multicore accelerators. The digital side handles operations and workloads that do not map naturally to the analogue arrays.
This seems less like a temporary compromise and more like a practical default architecture:
put each operation on the substrate that handles it well, while keeping one programmable system interface.
Mapping and compilation
A trained software model normally cannot be copied directly into an arbitrary physical substrate.
The deployment stack may need to decide:
- which operators are supported;
- which layers remain digital;
- how large matrices are divided across tiles;
- how signed weights are encoded;
- how many devices represent one logical weight;
- which numerical scales each tile uses;
- how partial sums are accumulated;
- where data is buffered;
- how communication is scheduled;
- how faulty devices are avoided;
- how calibration parameters are applied.
Legno, although aimed at programmable analogue dynamical systems rather than ordinary DNN accelerators, is an informative example: a compiler is required to turn a mathematical program into configured physical blocks while accounting for range, noise and device constraints.
The broader overview of software stacks for analogue in-memory accelerators describes weight stationarity, pipelining, device noise and heterogeneous execution as properties that make conventional deployment tools insufficient on their own.
In other words, electrical programmability is only the lowest layer of programmability. A reusable AI device also needs a stable mapping contract above it.
If the target is a Transformer or an LLM
A Transformer should probably be divided into several different hardware problems rather than treated as one uniform network.
Static model state
projection and feed-forward weight matrices
Dynamic sequence state
keys, values and intermediate activations
Global or nonlinear operations
Softmax, normalization, activation functions and reductions
System functions
tokenization, sampling, memory management, routing and control
Static linear weights
The large linear layers are the part most naturally suited to stationary in-memory weights.
A 2025 demonstration mapped the shared fully connected weights of ALBERT onto a 14 nm PCM analogue-inference chip. The ALBERT hardware paper is useful partly because it states the boundary clearly:
- the fully connected blocks were implemented in hardware;
- attention matrix computations, GELU, Softmax, Tanh, pooler and classifier operations remained in software;
- hardware-aware fine-tuning was used;
- recalibration mitigated conductance drift;
- average accuracy across the reported GLUE tasks was close to, but below, the floating-point reference.
This is a substantial proof of feasibility for Transformer components. It is not a demonstration that every part of a Transformer should be implemented by the same analogue mechanism.
Dynamic attention and KV state
Autoregressive generation adds another difficulty: keys and values are created while the model is running.
These states require a different memory profile from fixed model weights:
- fast repeated writes;
- adequate retention for the active sequence;
- high write endurance;
- rapid reads;
- suitable density;
- dynamic allocation and eviction.
A gain-cell in-memory attention architecture addresses this more directly by storing token projections in charge-based memories and computing attention dot products in the same arrays.
The paper also reports an important boundary: circuit constraints and non-idealities prevented direct mapping of ordinary pretrained models, so an adapted initialization method was required.
This suggests that static weights and dynamic KV state may need different physical technologies, control policies or training assumptions.
Communication
Large Transformers also involve substantial movement between:
- projection layers;
- attention heads;
- KV storage;
- feed-forward layers;
- residual paths;
- normalization;
- multiple chips or tiles.
Even when each matrix operation is efficient, the system can remain limited by movement and synchronization between them.
For this class of model, a plausible architecture is therefore heterogeneous:
analogue or spatial tiles
-> large stationary linear transforms
fast writable memory or specialized IMC
-> dynamic attention/KV data
digital vector units
-> normalization, nonlinear and reduction operations
host/runtime
-> allocation, scheduling, model control and generation logic
The exact split would depend on whether the intended device is for a small fixed language model, an edge model, a server-scale LLM, or something structurally different from a Transformer.
How to evaluate the idea fairly
It may help to separate three claims:
- the physical primitive performs its local operation efficiently;
- a complete layer is efficient;
- an end-to-end application is efficient at comparable quality.
A strong result at one level does not automatically establish the next.
System metrics
At minimum, an end-to-end comparison could include:
| Category |
Possible metrics |
| Model result |
Task accuracy, loss, perplexity or application-specific quality |
| Speed |
Single-input latency, sustained throughput and batch dependence |
| Energy |
Energy per inference or per generated token |
| Physical cost |
Silicon area, FPGA resources, memory and converter area |
| Deployment |
Programming, mapping and calibration time |
| Stability |
Repeatability, drift, temperature sensitivity and failure rate |
| Flexibility |
Supported operators, model sizes and update frequency |
Peak operations per second can be useful, but it should be accompanied by utilization and system-level measurements. Otherwise, converter, communication or unsupported-operation costs can remain hidden.
Evidence levels
It is also useful to label what kind of evidence a number represents.
Fabricated silicon running an end-to-end task
Measured device or array plus modeled system components
Post-layout or transistor-level simulation
Architecture-level simulation
Algorithmic simulation with hardware non-idealities
Ideal arithmetic count or peak throughput estimate
All of these can be useful. They simply answer different questions.
Baselines
A particularly useful control is a digital model using the same precision as the physical one.
Without that baseline, an improvement could come from several different changes at once:
- lower numerical precision;
- fixed-function specialization;
- weight locality;
- physical analogue computation;
- pruning or model simplification;
- a different task-quality target.
A three-way comparison can separate them more clearly:
Floating-point software reference
Matched-precision digital quantized implementation
Physical-aware simulation or physical implementation
The same model, dataset and quality target should be used where possible.
It is also worth reporting the physical core separately from the complete system, as long as the distinction remains explicit.
A small way to test the idea
One low-risk experiment would be to use a small network whose behaviour is easy to inspect, rather than beginning with a complete modern LLM.
For example:
Input
-> small dense layer
-> simple activation
-> second dense layer
-> output
Possible stages:
Stage 1: software reference
Train an ordinary floating-point model and preserve:
- weights;
- activations on a small test set;
- output quality;
- layer dimensions;
- operation counts.
Stage 2: matched digital quantization
Quantize weights and activations to the precision expected from the physical implementation.
This reveals how much quality is lost through low precision alone.
Stage 3A: FPGA spatial implementation
Use hls4ml or a similar flow to compare several reuse factors.
That gives a concrete curve between:
- maximum parallelism;
- latency;
- multiplier/DSP use;
- LUT and memory use;
- routing and synthesis feasibility.
This is a useful test of the “one operation per physical unit” side of the concept without requiring a custom analogue device.
Stage 3B: analogue-aware simulation
Use AIHWKit or CrossSim to add:
- weight-programming error;
- finite input/output precision;
- read noise;
- drift;
- array-size limits;
- parasitic effects where supported.
This identifies which physical assumptions the model is sensitive to.
Stage 4: lifecycle checks
If model replacement is part of the concept, measure it independently from inference:
program model A
verify and calibrate
run reference inputs
program model B
verify and calibrate
run reference inputs
restore model A
compare the original and restored outputs
That small cycle can reveal:
- write repeatability;
- history dependence;
- recalibration requirements;
- retained accuracy;
- replacement latency.
Stage 5: expand only the boundary that remains promising
Depending on the result:
Arithmetic dominates and programming is stable
-> increase the number of physical layers
Converters dominate
-> share, simplify or move the conversion boundary
Routing dominates
-> use larger tiles, serialization or structured connectivity
Drift dominates
-> add recalibration or hardware-aware training
Model replacement dominates
-> keep a fixed physical base and update a smaller digital component
Dynamic state dominates
-> separate fixed weights from fast writable state
Network is naturally sparse and temporal
-> consider the neuromorphic branch
This would produce useful information even if the final architecture turns out not to be a fully physical neural network.
Overall, the idea seems to have several technically credible roots. The most mature route is currently to specialize a well-chosen boundary—often a matrix/tile or a compact fixed pipeline—rather than requiring every operation and control function to use the same physical mechanism.
The central design decision would therefore be both a compute boundary and a control boundary:
What is physically instantiated, what remains programmable in digital logic, and what must be observable and recalibratable for the device to remain interchangeable at scale?
Once that boundary is chosen, the relevant existing field—and the smallest useful prototype—becomes much easier to identify.
