How QSO is validated

QSO makes no claim it cannot test. This page lists the kinds of evidence behind its results and where to find them.

References for every algorithm

Every algorithm is checked against an independent reference:

Kind of reference	Examples
Closed forms	Grover’s sin²((2k+1)θ); QAOA on rings, (2p+1)/(2p+2); the one-layer QAOA formula of Wang et al.; walk limits such as 145/169; T₁/T₂ decay; n! bunching
Published values	0.6925 and 0.7559 for QAOA on large-girth cubic graphs; the full-CI energy of H₂/STO-3G; automorphism group orders
Dense matrices	every gate, the QFT against the DFT matrix, Lanczos and Krylov against full diagonalisation
A second method	adjoint, parameter-shift and finite-difference gradients; statevector against density matrix
Other simulators	Qiskit Aer, Qulacs, Cirq, PennyLane Lightning and The Walrus

These checks live in the test suites, which run on every change on Linux (x86_64 and arm64), macOS and Windows:

Rust: unit, validation, property and robustness tests;
Python: tests of the bindings.

Cross-validation with other frameworks

benchmarks/cross_framework compares QSO with established simulators. It runs every week, and a deviation above rounding level fails the run.

Comparison	Largest difference
Final states of random circuits (Qiskit Aer, Qulacs, Cirq, PennyLane Lightning)	2.4·10⁻¹⁵
Energies and gradients of a 100-parameter ansatz (PennyLane adjoint, Qulacs backprop)	1.8·10⁻¹⁵
Density matrices of noisy circuits (Qiskit Aer)	3.5·10⁻¹⁵
Permanents (The Walrus)	2.8·10⁻¹⁴ relative

Robustness

Check	What it covers
Fuzzing	Each parser and deserialiser runs under cargo-fuzz for one minute on every pull request and ten minutes every week. The parsers are OpenQASM, Pauli sums and graph specifications; the JSON forms are those of circuits, states, graphs and Pauli sums. Beyond “no crash”, the targets check that exported QASM re-imports to an equivalent circuit and that accepted data are well-formed. Every input that ever failed is replayed on each run.
Miri	The `unsafe` statevector kernels run under Miri on multi-threaded paths, which checks them for undefined behaviour and data races.
Hostile input	Tests feed oversized, non-finite and inconsistent input to every public entry point and expect an error.

API stability

QSO 1.x adds and never breaks:

cargo-semver-checks compares the public Rust API with the last release;
stubtest checks that the Python type stubs match the compiled module.

Performance

The benchmark report states the machine it ran on. A weekly run compares the timings with the median of the previous four runs. A slowdown of more than 25 % is flagged.

The technical note

Validated Quantum Algorithm Benchmarks of the Metatron QSO Simulation Framework (DOI 10.5281/zenodo.23623226) reports all of the above with the data and the programs that produce it.

Keyboard shortcuts