The same quantum program can become a different physical experiment

An October 15 preprint compares how identical source circuits are compiled and executed across several commercial quantum devices. The study includes preregistered questions and later exploratory amendments, and finds that placement and device generation can change observed divergence orderings. Its evidence supports conditional comparisons rather than a universal provider ranking. The practical issue is that a source program specifies a logical task, while the compiled gates, physical locations, reference distribution and available interface information determine which experiment was actually performed.^[1]

The experiment covers three vendors and four devices. A comparison between eight- and twelve-qubit divergence reverses between IQM Emerald and Garnet, illustrating why device generation belongs in the result. The paper separates preregistered analyses from amendments and exploratory work. Two repetitions can establish a specified recurring ordering without demonstrating a long-term ranking of providers.^[1]

An original routing example uses a four-site line A–B–C–D. A logical interaction between states initially at A and D is not a native nearest-neighbor operation. One illustrative implementation moves the state at A toward C, applies the interaction with D and restores the original placement. Restoring it creates additional work that might be avoided if later operations accept the changed mapping. This is not a minimum-gate proof or a benchmark result. It shows why the future circuit and the placement policy affect the cost of the same logical interaction.

Native gate definitions can also evolve. Two device generations from one provider can expose different entangling operations, calibrations or compilation options. A software interface may summarize capabilities differently from the pulse-level implementation. Comparing compiled gate counts requires checking what each counted gate means and whether the interface describes the actual execution path. Otherwise, a numerical difference can reflect a changed reporting convention as much as a changed physical resource. Versioned artifacts help preserve the meaning of the comparison when the service later changes.

Execution divergence needs a reference. A measured output distribution can be compared with the ideal distribution, with another device or with repeated runs on the same device. Those comparisons answer different questions. Two devices can agree with each other while sharing the same systematic error, or disagree because one is closer to the ideal target. The metric should therefore identify both distributions and include sampling uncertainty. A single distance value without that context can make disagreement look like a direct measure of overall hardware quality when it is not.

Statistical interpretation matters especially for provider orderings. Failure to find a significant difference does not establish equivalence. The experiment may simply lack enough precision to resolve the difference that matters. Conversely, a statistically resolved difference under one placement and workload does not prove a global ranking across applications. A useful equivalence question specifies an acceptable performance gap in advance and asks whether the data constrain the difference inside it. That is a stronger and more informative statement than reporting that two error bars overlap.

Preregistration clarifies which questions and analyses were fixed before results were seen. Exploratory amendments can still produce valuable discoveries, provided their timing and motivation remain visible. They should guide subsequent confirmation rather than being presented as though every detail was predetermined. This distinction is particularly useful in hardware studies, where unexpected routing behavior or interface changes can justify new experiments. The evidence becomes stronger when the record separates the original test, the observation that prompted a revision and the independently repeated check of the revised explanation.

The study points toward an operational definition of portability: a program can run across platforms while its cost and output behavior remain conditional on the physical realization. The next useful benchmark would publish those conditions in a reproducible bundle and repeat the comparisons after controlled placement or device changes. Readers could then identify whether a difference follows the algorithm, embedding, compiler or hardware. That diagnosis is more actionable than a broad league table because it tells a developer which part of the execution path must change to improve the specific task.