Watch it design a chip
Llama 3.2 1B decoding at 5 tokens per second, within 8 mm² and 400 mW. The agent starts from a bare microcontroller, names what limits it, tries every single change, keeps the best and says why. The floorplan is drawn to scale at every step. This run evaluated 2,154 designs in under 10 seconds.
- Start
Start from a bare microcontroller
A CV32E20 at 100 MHz with two 32 KiB banks on IHP 130 nm. The CPU does every multiply itself, and the 642 MiB of weights and KV cache do not fit in a 64 MiB HyperRAM.
0.0097 tokens per second
- Decision 1
Move to 65 nm, 200 MHz and LPDDR2
A newer node allows faster clocks and memory interfaces at lower energy. Before this step 100% of the time went to compute.
0.0097 to 0.019 tokens per second, 854 to 327 mJ per token, 3.53 to 2.92 mm², now buildable
- Decision 2
Add a GEMV engine
Matrix multiplies dominate and the CPU does them one MAC at a time. Before this step 100% of the time went to compute.
0.019 to 1.70 tokens per second, 327 to 86.2 mJ per token, 2.92 to 3.47 mm²
- Decision 3
Move to 28 nm, 400 MHz and LPDDR4X
A newer node allows faster clocks and memory interfaces at lower energy. Before this step 75% of the time went to the accelerator's ports.
1.70 to 3.30 tokens per second, 86.2 to 29 mJ per token, 3.47 to 2.43 mm²
- Decision 4
Move to 16 nm, 800 MHz
A newer node allows faster clocks and memory interfaces at lower energy. Before this step 76% of the time went to the accelerator's ports.
3.30 to 6.18 tokens per second, 29 to 27.9 mJ per token, 2.43 to 1.34 mm², meets every target
- Decision 5
Switch to LPDDR5
A slower interface can be cheaper in area and energy per bit. Before this step 78% of the time went to the accelerator's ports.
throughput unchanged at 6.18 tokens per second, 27.9 to 22.5 mJ per token, 1.34 to 1.00 mm²
- Decision 6
Make the banks 16 KiB
Unused SRAM costs area and leakage. Before this step 78% of the time went to the accelerator's ports.
throughput unchanged at 6.17 tokens per second, 1.00 to 0.97 mm²
No single change lowers the energy further, so the agent stops with a 4 x 8 GEMV engine next to a CV32E20 at 800 MHz in 16 nm, fed by LPDDR5: 6.17 tokens per second at 22.5 mJ per token in 0.97 mm². Its weights now arrive as fast as the accelerator's ports take them in, so more ports would raise the token rate but not lower the energy per token.

