← Back to blog

The complexity cliff, and how ε stays on top of it

We measured CopperPilot ε against a general-purpose agent on the same LLM: 341 real KiCad tasks across 49 designs, graded by KiCad rather than an AI judge.

CopperPilot epsilon compared with a general-purpose agent using the same LLM across end-to-end PCB design stages

In engineering, ε is the small gap you are always trying to close: between the spec and the measurement, or between the design you meant and the board you got. We named our design agent after it because that gap is the whole job.

We are not announcing ε today. Before we say more about it, we wanted to know exactly how good it is. So we measured it.

We built a ruler first

“AI designed a PCB” demos are easy to make and hard to trust. So we built an internal benchmark of 341 real KiCad tasks drawn from 49 designs. They range from a USB-C breakout with a handful of parts to an 8-layer Jetson Nano carrier with 370 components.

Every task follows a real stage of the design flow: schematic capture, board setup, placement, copper pours, routing, 3D constraints, and full end-to-end builds from an empty project.

Grading is done by KiCad itself: ERC, DRC, and netlist export, plus scripted checks on the design files. No AI judge decides the score.

Same LLM, different agent

To isolate what the agent contributes, we ran ε and a general-purpose agent on the same LLM under the hood, with the same tasks and limits. The difference in the chart is the agent.

Across all 341 tasks, ε scores 83.2 versus 54.7. It wins or ties 320 of 341 head-to-head tasks, and scores a perfect 100 on 197 tasks versus 91 for the general-purpose agent.

The largest gaps appear where an agent must create structure rather than edit something that already exists. On full end-to-end board builds, ε scores 78.0 versus 18.6. On schematic capture, 83.3 versus 36.4. On layout, 82.7 versus 48.7. The gap narrows on routing and 3D work, where both agents start from a board that already has more of its structure in place.

What happens as boards grow

A 300-component board is not a bigger 10-component board; it is a different problem. Constraints pile up: which nets matter, which parts must sit together, what the stackup allows, and which earlier decision still controls the current step.

The general-purpose agent loses about 8 points of score every time component count grows 10×. On the largest designs, above 100 components, it averages about 35/100 and completes none of the end-to-end builds.

ε stays much flatter. It averages 86–89 from 10 to 100 components and 73 above 100. On those largest designs, its end-to-end builds score 97/100.

The operational difference is just as telling. The general-purpose agent uses about twice the tool calls and twice the input tokens per task, yet times out 32 times. ε times out four times.

Why ε holds on longer tasks

ε is built for PCB design. It keeps the parts of the design that matter for the current step in focus and carries design intent across long sessions. A constraint established during planning must still be respected during placement and routing. A net that mattered in the schematic cannot disappear when the board gets crowded.

It also has a built-in research engine that can look up datasheets and reference designs the way an engineer would. The benchmark results are consistent with that combination: steadier scaling, fewer timeouts, and roughly half the token use.

Measuring like scientists

This is one evaluation seed. More runs are underway, and we will keep reporting where ε falls short as well as where it wins. Routing on the largest boards is still hard for every agent we have tested, ε included. That is where our ε is largest, and it is what we are working on next.

Stay tuned for open source PCB releases soon!

More from the blog