November 17, 2026
One program to drive them all, PennyLane to define them, Grace Hopper to decode them all, and Backline to bind them.
What does it take to run a Backline program on a different machine, with a different RDMA engine on the FPGA? Only a few settings on the controller and the coprocessor. Backline now runs on the NVIDIA GH200 Grace Hopper™ Superchip and, alongside AMD ERNIC™ and NVIDIA's open-source Holoscan Sensor Bridge, it supports ETH Zurich's open-source BALBOA RDMA engine — so neither the coprocessor nor the RDMA engine ties you to one vendor, and your quantum program stays the same.
Since we announced Backline with AMD in September, the question we hear most often from groups building quantum control systems is a simple one — will it work with our hardware?
It's a fair question. Control systems for different qubit technologies are built from very different classical hardware, with different FPGAs, network cards and host machines, and with different rules about which IP a team can license or change. For Backline to be useful across the ecosystem, it has to meet each group where its hardware already is.
In our last post, we added NVIDIA's Holoscan Sensor Bridge (HSB) to Backline, and showed how much the RDMA engine on the FPGA can change the round trip. Today, we're changing the other end of the link too. Backline now runs on Grace Hopper, with its Arm-based Grace CPU or its H100 GPU answering each round, and it supports a third RDMA engine, ETH Zurich's BALBOA.
So, whichever coprocessor and RDMA engine you choose — or bring yourself — your quantum program stays the same.
Read on to see a full Backline program running on Grace Hopper, how the three RDMA engines fare there, and how to bring your own.
What's new?
Here's a complete Backline program, with an AMD Versal™ VPK120 FPGA as the controller, Grace Hopper's H100 GPU as the coprocessor, and BALBOA as the RDMA engine:
import pennylane as qp
FPGA = qp.Controller(
name="fpga-controller",
remote=True,
hardware="fpga",
backend="balboa_hwhs", # "hsb_hwhs" for HSB, or "hwhs" (default) for ERNIC
executor_options={**FPGA_SERVER, "port": 9811},
)
GPU = qp.Coprocessor(
name="gpu-coproc",
coprocessor_fn="gpu_steane_launcher",
remote=True,
hardware="gpu", # the H100
endpoint=qp.Endpoint(GH200_ADDRESS, 7760), # an address the board's Arm cores can reach
executor_options={**GH200_SERVER, "port": 9813},
# peer=hsb for HSB, or peer=catalyst (default) for ERNIC
init_args={"config": "dev=mlx5_1;gid=5;peer=balboa"},
)
dev = qp.Backline(controller=FPGA, coprocessors=[GPU], transport="rdma", qec_code="steane")
@qp.qjit(capture=True)
@qp.set_shots(1000)
@qp.qnode(dev, mcm_method="one-shot")
def ghz():
qp.Hadamard(0)
qp.CNOT([0, 1])
qp.CNOT([1, 2])
return qp.sample([qp.measure(0), qp.measure(1), qp.measure(2)])
Here, FPGA_SERVER and GH200_SERVER hold the connection details for the FPGA board and Grace
Hopper server respectively. Simply load the BALBOA bitstream (the FPGA design available via
the backline-vpk120 repository) onto the board,
and set the RDMA engine in two places — the controller’s backend argument, and the coprocessor's
peer, which tells it which RDMA engine sits on the other end of the link.
As a reminder from our last post, the RDMA engine is the block of logic on the FPGA that turns data into network packets and back, using RDMA over Converged Ethernet (RoCE v2). In addition to AMD's ERNIC and NVIDIA's HSB, Backline now supports ETH Zurich's BALBOA. Like ERNIC, it uses reliable connections, and like HSB, it's open source (under the BSD 3-Clause and MIT licenses).
Three RDMA engines on Grace Hopper
How does this work (and perform!) in practice? To find out, we joined the VPK120 FPGA to an NVIDIA ConnectX®-7 network card in the GH200 by a single direct attach cable, with no switch in between, as in our last post. We then loaded each RDMA engine's bitstream in turn, and used Backline's benchmark harness to run a million rounds of an echo — where the coprocessor sends each request straight back without decoding it — with first the Grace CPU and then the H100 GPU answering each round.
On Grace Hopper, the RDMA engine you pick changes the round trip by about 0.5 µs, whether the Grace CPU or the H100 GPU answers each round.
HSB has the lowest round trip of the three to both coprocessors, while BALBOA and ERNIC sit level on the median (the difference between those two instead shows up in the tail — ERNIC's is much longer than BALBOA's).
| Coprocessor | RDMA engine | Median (µs) | 99.999th pct. (µs) | Rounds above 3 µs (CPU) / 6 µs (GPU) |
|---|---|---|---|---|
| Grace CPU | HSB | 2.10 | 2.23 | 1 |
| Grace CPU | BALBOA | 2.64 | 3.22 | 26 |
| Grace CPU | ERNIC | 2.66 | 3.95 | 668 |
| H100 GPU | HSB | 4.5 | 5.5 | 2 |
| H100 GPU | BALBOA | 5.0 | 5.5 | 0 |
| H100 GPU | ERNIC | 5.0 | 6.3 | 75 |
The open, swappable stack
With the addition of HSB and BALBOA to the PennyLane, Catalyst, and Backline stack, the RDMA engine is no longer a black box — you can read it, change it, or replace it with your own.
And this is critically important! The network stack affects more than the median latency. It also sets the tail of the latency distribution, how many FPGA resources you use up, which licenses apply to your design, and whether you can open the stack up and change it. Different needs require different choices, and Backline should work with all of them.
Not only that, but with HSB or BALBOA in the bitstream, every layer from your PennyLane program down to the RDMA engine on the FPGA can be open source (depending on your system's configuration!). With ERNIC, the RDMA engine itself is AMD's licensed IP.
Which RDMA engine should I use?
Each RDMA engine makes a different trade-off, so you can pick the one that suits your experiment:
| If you need... | Use | Controller backend | Coprocessor peer |
|---|---|---|---|
| The shortest round trip over a direct attach cable | NVIDIA HSB (Apache 2.0) | hsb_hwhs | peer=hsb |
| Reliable connections from an open-source engine | ETH Zurich BALBOA (BSD 3-Clause, MIT) | balboa_hwhs | peer=balboa |
| Reliable connections or many producers, with AMD IP already in many Versal designs | AMD ERNIC | hwhs (default) | peer=catalyst (default) |
| Something none of these engines does | Your own | The name in your manifest | peer=catalyst, or your own coprocessor backend |
Want to bring your own RDMA engine? Backline supports that as well. Build it into your bitstream behind Backline's hardware handshake, as we did for HSB and BALBOA, then write a library that implements Backline's controller interface. The name and settings for the engine live in a small JSON manifest — that name is what you pass as the controller's backend.
For more details, see Catalyst's Backline developer documentation.
Try it out and get involved
Backline now runs on Grace Hopper with three RDMA engines to choose from on the FPGA. Whichever coprocessor and RDMA engine you choose, your quantum program stays the same.
We're excited to see which RDMA engines you use with Backline, whether it's one of the many designs already out there, or one you've built for your own control system.
Ready to try Backline on Grace Hopper? Here's where to start:
-
Get the FPGA designs: Backline's HSB and BALBOA designs for the VPK120 are in the backline-vpk120 repository. BALBOA itself comes from ETH Zurich's fpga-network-stack, and HSB comes from NVIDIA's Holoscan Sensor Bridge project.
-
Explore the technology: Read the Backline paper for a technical overview of the architecture and its original measurements.
-
New to Backline? Read the announcement, and see how to go from local CPU prototyping to low-latency FPGA-GPU communication in our demo. Then bring your own decoder, and see how much the RDMA engine can change the round trip latency.
Have hardware you'd like to see running with Backline? Let us know! Head over to our GitHub repository to check out the ongoing work and join in on the development, or send us an email at backline@xanadu.ai to tell us how you are using PennyLane, Backline, and Catalyst in your workflows.
And if you are as excited as we are, make sure to sign up for the PennyLane newsletter for the latest PennyLane features and updates!
About the authors
Hong-Sheng Zheng
Hong-Sheng Zheng is a compiler engineer on the Compilation team at Xanadu.
Ali Farazdaghi
Ali is a systems software developer focused on performance, low-latency communication, GPUs and compute accelerators.
Joseph Lee
Joseph is a physicist and software developer. At Xanadu, he leads the performance team and makes PennyLane go zoom.
Lee O'Riordan
Physicist, purveyor of angular momentum, GPUs, pointy guitars, and computational things. Working on quantum stuff.
Josh Izaac
Josh is a theoretical physicist, software tinkerer, and occasional baker. At Xanadu, he contributes to the development and growth of Xanadu’s open-source quantum software products.