Performance Benchmarks · V4.3
Smaller models,
faster CPUs and GPUs,
smaller silicon
PNN keeps every layer full rank but sparse: each output reads a fixed set of inputs, the same number for every output, and no index is stored. Every figure below is measured unless it is labelled an estimate — from 32-bit floats down to 1-bit weights, on CPUs, on NVIDIA GPUs, on a phone NPU, on an FPGA and through silicon place and route.
Compare vs NVIDIA →CPU inference · measured
Faster than dense int8 on four CPU families
The PNN CPU runtime ships as source and pre-built libraries for x86-64 (AVX2, AVX-512) and ARM64 (NEON), and picks the fastest kernel when it loads. Its kernels use SLNG — single load, no gather: one ordinary vector load where other sparse kernels need a gather, no index lists, and the same multiply instructions as dense. Only the weights that exist are computed, so the speed-up tracks the density.
Chart: a 4096 × 4096 int8 layer, batch 1, one thread, the delivered runtime against a tuned dense int8 kernel on the same CPU. Whole-model figures: one ViT-B/16 image (197 tokens) through the 48 block layers on an Intel Core i3-8100. 4-bit weights halve the weight memory on CPUs; they do not make CPU inference faster.
GPU inference · measured
Faster than dense on NVIDIA GPUs, token by token
Generating one token at a time, a GPU spends its time reading weights, not multiplying them. PNN's GPU kernels use SLNG, as on the CPU: they read only the weights that exist, with no index lists, so a PNN model generates faster than the same model dense at the same precision. Measured on a decoder with the shape of a 7-billion-parameter language model, every linear layer in PNN, on three NVIDIA generations.
Chart: one decoding step at one token, fp16, a 4-layer decoder (hidden size 4096, 32 heads, feed-forward width 11008, key–value cache of 512 tokens) captured as one CUDA graph; every linear layer PNN, attention the standard kernel, against the same model with every layer dense on cuBLAS. A full-depth model repeats the same layers. Single layers: 4096 × 4096, 4096 → 11008 and 11008 → 4096 against cuBLAS fp16 and cuBLASLt int8. With many tokens per step — prompt processing, batched serving, training — the GPU is compute-bound and dense tensor-core GEMM wins; there PNN layers run as dense, at dense speed, while the model is still stored at its low density. The GPU kernels are not yet part of the evaluation package.
Phone NPU · Snapdragon 8 Elite · cycle-accurate simulation
Twice the tokens per second on a Snapdragon NPU
Llama-3.1-8B generating text on the Hexagon NPU of the Snapdragon 8 Elite, every linear layer in PNN at 25 % density with 4-bit weights. A PNN kernel on the NPU's vector units generates 1.8–2.2× faster than Qualcomm's own dense result — in software, on the phone's existing silicon — and the model's weights take a quarter of the memory of dense 4-bit.
PNN kernel cycles from Qualcomm's cycle-accurate Hexagon simulator (Hexagon SDK 6.7, v79), with the memory speed measured on a Galaxy S25 through Qualcomm AI Hub. The NPU clock (1.0–1.5 GHz) is assumed until the kernel is timed on a device. Dense: Qualcomm's published Genie result for Llama-3.1-8B (4-bit weights, 16-bit activations, on the NPU's matrix unit), its production path. One token at a time.
Vision transformer · ImageNet-1k
ViT-B/16 from 346 MB to 8.9 MB
All 48 linear layers of the 12 transformer blocks become PNN layers at 50 % density; the model is converted from the public timm checkpoint and trained for 10 epochs. The same PNN model then goes to INT8 for 0.32 points without retraining, to 4-bit weights, and — with 10 more epochs — to 1-bit weights at 71.19 %, 39× smaller than the dense file and still gaining 1.8 points in its last epoch.
Top-1 on all 50,000 ImageNet-1k validation images. Bars to scale (MB, linear). INT8 and INT4 are post-training quantisation of the FP32 PNN model, no retraining. 1-bit: each connection is +1 or −1 with FP16 scales (0.53 bits per weight); patch embedding, layer norms and head stay FP16. The FP32 to INT4 PNN models and the 1-bit model are published on Hugging Face: FP32 · BF16 · INT8 · INT4 · 1-bit (8.9 MB).
New · 1-bit PNN
A vision transformer in 8.9 MB — no multipliers
In a 1-bit PNN layer every connection is a sign. Each output adds or subtracts the inputs it reads — half of them at 50 % density — and is scaled once. The whole model fits in on-chip memory, so inference needs no DRAM traffic for weights, and the processing element needs no multiplier.
Against NVIDIA Jetson on ViT-B/16
A 1-bit PNN 50 % chip sized to run ViT-B/16 10× faster than the fastest published Jetson result needs a 12 mm² die at 16 nm drawing 0.6 W.
| ViT-B/16, batch 1 | Jetson Orin Nano | Jetson AGX Orin | 1-bit PNN chip, 16 nm (est) |
|---|---|---|---|
| Speed | 59 images/s | 43 images/s | 590 images/s |
| Latency per image | 17.0 ms | 23.5 ms | 1.7 ms |
| Energy per image | ≤ 425 mJ | ≤ 940 mJ | 1.0 mJ |
| Power | 25 W mode | 15–40 W | 0.6 W |
| Size | 70 × 45 mm module | 100 × 87 mm module | 12 mm² die |
| Weights | DRAM | DRAM | all on-chip |
| ImageNet-1k top-1 | 84.5 % (dense) | 84.5 % (dense) | 71.19 % |
Jetson: published latencies (Orin Nano 17.0 ms, TensorRT 10.7, 25 W mode, arXiv 2512.09792; AGX Orin 23.5 ms, arXiv 2512.20120); energy = power mode × latency, an upper bound. PNN chip: modelled at 16 nm from the measured area and power ratios of the 1-bit element, attention on a small INT8 engine — a model, not silicon (node data ±40 %). It gives up accuracy against the dense model; in exchange it fits in a camera, a sensor head or a battery-powered device.
Hardware · measured
Processing elements from 8 bits to 1 bit
The PNN processing element is generated at 8-, 4- and 1-bit weights and taken through synthesis, place and route in sky130 with the same flow and clock. Everything around the arithmetic stays the same, so the ratios compare directly.
| Processing element | Area | Power |
|---|---|---|
| 8-bit weights (reference) | 1.00× | 1.00× |
| 4-bit weights | 0.50× | 0.37× |
| 1-bit weights | 0.17× | 0.07× |
OpenLane / sky130, 25 ns clock, all designs meet timing. Power is the tool's estimate at its default switching activity — a fair comparison between designs, not an absolute figure for an advanced node.
- Dual-mode multiplier. One byte holds one int8 weight or two int4 weights; the unit computes one 8-bit or two 4-bit products per cycle, and is formally proven equal to both definitions.
- 4-bit and 1-bit inference on an FPGA. The generated inference cores run on a Xilinx Artix-7 board (Digilent Arty A7-100T): 24,000 outputs (4-bit) and 28,000 outputs (1-bit) equal to the bit-exact model. The 1-bit core has no multipliers at all.
- Inference and training on FPGA. PNN inference and on-chip training verified on Artix-7 — 17 % of the chip's logic, no DSP blocks.
Results across tasks
Summary — 5 seeds, iso-architecture
| Task | Metric | Dense | PNN | PNN int8 | Savings |
|---|---|---|---|---|---|
| MNIST | acc % ↑ | 97.45 | 97.02 | 97.00 | 4.5× fewer · 18× smaller int8 |
| Fashion | acc % ↑ | 88.57 | 88.17 | 88.14 | 4.5× · 18× |
| Tabular* | acc % ↑ | 88.71 | 87.65 | 87.38 | 2.4× · 9.6× |
| NLP-GPT | val loss ↓ | 2.268 | 2.387 | — | 1.7× · 6.9× |
| Waymo BEV occ.† | acc % ↑ | 93.35 | 93.71 | — | 6.7× fewer (width 4096) |
| Waymo real LiDAR‡ | acc % ↑ | 92.92 | 92.92 | — | 6.67× (width 4096) |
| microGPT | op-bound speed | 1.0× | 1.59× faster | — | scalar / no-BLAS |
↑ higher = better, ↓ lower = better. NLP metric is validation cross-entropy loss. * Tabular = recommended fc2-only config. † Waymo-class BEV occupancy (synthetic), 3-seed, width 4096 — savings grow with width (below). ‡ Real Waymo Open Dataset v2 LiDAR, occupancy forecast +0.5 s: dense and PNN identical to every printed digit — exact parity. INT8 lossless on vision.
Scaling — perception
Savings grow with width
The connectivity gets structurally sparser as the layers widen, so on a Waymo-class BEV occupancy task the parameter savings rise from 4.0× to 6.7× as width goes 512 → 4096 — while accuracy stays at parity or better (negative gap = PNN ahead). The wider the model, the more PNN saves.
Validated on real Waymo LiDAR: on real Open Dataset v2 point clouds (occupancy forecast, +0.5 s, ego-motion compensated) dense and PNN score identically to every printed digit — 92.92% acc, 0.587 IoU — at both 4.93× (width 1024) and 6.67× fewer parameters (width 4096). The dense model's extra 43 M parameters buy nothing.
| Layer width | Connectivity density | Params saved | Accuracy gap |
|---|---|---|---|
| 512 | 22.9% | 4.05× | +0.9 pp |
| 1024 | 19.3% | 4.93× | −0.1 pp |
| 2048 | 16.6% | 5.85× | −0.12 pp |
| 4096 | 14.7% | 6.67× | −0.36 pp |
Waymo-class BEV occupancy (perception-shaped), 3-seed mean, iso-architecture. Accuracy gap = dense − PNN. At width 4096 the PNN is smaller, slightly more accurate, and faster per epoch than the dense baseline.
Parameters
Fewer parameters per model
Parameter-reduction factor: dense ÷ PNN active parameters, iso-architecture (higher = better). Waymo LiDAR = real Open Dataset v2, width 4096 (50.5 M dense vs 7.6 M PNN).
Accuracy
Near-parity, shown truthfully
Waymo LiDAR (real data, occupancy forecast +0.5 s): dense and PNN are identical to every printed digit. Test accuracy %, dense vs PNN side by side, y-axis from 0 so the small gap is shown honestly. Mean of 3–5 seeds.
Memory
Model footprint — dense FP32 vs PNN INT8
Megabytes, log scale. Index-free connectivity means connections are computed, not stored.
Scaling — language · concept
GPT-3 on one die
GPT-3's widest layers are PNN's best regime: connectivity density falls to 10–12%, cutting 175 B parameters to 19.3 B active (9.08×). That is exactly the factor that makes the impossible chip possible — all of GPT-3, weights resident on a single 2 nm reticle-limit die, no HBM. The dense model needs ~7 reticles at the same precision.
Analytical estimate from a reproducible calculator (INT4 weights in mask ROM, KV on-die, ctx 512, 16.7 TOPS/W sustained); silicon figures are engineering estimates pending back-annotated synthesis. Accuracy at GPT-3 width is the evaluation-phase measurement — parity is demonstrated to width 4096.
Honest regime map
Where PNN wins — and where it doesn't
Wins
- • Memory: always — 4.5× fewer params (up to 6.7× on wide perception); ViT-B/16 from 346 MB to 49 MB at INT8 and 8.9 MB at 1 bit.
- • Accuracy: parity on real Waymo LiDAR, near-parity on vision, graceful on tabular (≤1.1 pp).
- • CPU inference (measured): faster than tuned dense int8 on Intel, AMD, Arm and Apple CPUs — 2.5–4.6× at 25 % density, one thread; bit-exact.
- • GPU inference (measured): faster than dense at the same precision in token-by-token generation — a 7B-shaped decoder at 25 % density runs 2.1–3.4× faster at one token on NVIDIA T4, L4 and H100, and stays faster up to 8 tokens.
- • Phone NPU (cycle-accurate simulation): Llama-3.1-8B at 25 % density, 4-bit, generates 1.8–2.2× faster than Qualcomm's dense result on the Snapdragon 8 Elite, with a quarter of the weight memory.
- • Low precision: INT8 for 0.32 points on ViT-B/16 without retraining; 4-bit and 1-bit weights supported from model file to silicon.
- • Runs on FPGA (measured): 4-bit and 1-bit inference bit-exact on Artix-7; inference and on-chip training in 17% of the chip's logic, no DSP blocks.
- • Silicon (place & route): a 1-bit element at 0.17× the area and 0.07× the power of the 8-bit one.
Limitations
- • Batched GPU work is compute-bound: with many tokens per step — prompt processing, batched serving, training — dense tensor-core GEMM wins, so PNN layers run as dense there, at dense speed. On the phone NPU the speed-up comes from PNN's own kernel, not the stock toolchain, and stays a simulation until timed on a device.
- • Transformers lose accuracy at 50 % density on a short schedule: ViT-B/16 78.45 % vs 84.51 % dense after 10 epochs; 1-bit 71.19 %.
- • 4-bit weights halve memory on CPUs but do not make CPU inference faster.
- • INT8 is not lossless everywhere — lossless on vision, harmful if input layers are over-sparsified.
- • Custom-silicon (ASIC) numbers are engineering estimates — the chip is not yet taped out.