Non-deterministic behaviour with fixed seed

[cqc-alec]

Version: 0.6.0.dev6

The following code uses an error model to simulate a Bell circuit, with a fixed seed. It gives non-deterministic results.

from pecos.error_models.generic_error_model import GenericErrorModel
from pecos.engines.hybrid_engine import HybridEngine

phir = """{
    "format": "PHIR/JSON",
    "version": "0.1.0",
    "ops":
    [
        {
            "data": "qvar_define",
            "data_type": "qubits",
            "variable": "q",
            "size": 2
        },
        {
            "data": "cvar_define",
            "data_type": "i64",
            "variable": "c",
            "size": 2
        },
        {
            "qop": "H",
            "angles": null,
            "args":
            [
                [
                    "q",
                    0
                ]
            ]
        },
        {
            "qop": "CX",
            "angles": null,
            "args":
            [
                [
                    [
                        "q",
                        0
                    ],
                    [
                        "q",
                        1
                    ]
                ]
            ]
        },
        {
            "qop": "Measure",
            "returns":
            [
                [
                    "c",
                    0
                ]
            ],
            "args":
            [
                [
                    "q",
                    0
                ]
            ]
        },
        {
            "qop": "Measure",
            "returns":
            [
                [
                    "c",
                    1
                ]
            ],
            "args":
            [
                [
                    "q",
                    1
                ]
            ]
        }
    ]
}
"""

error_model = GenericErrorModel(
    error_params={
        "p1": 2e-1,
        "p2": 2e-1,
        "p_meas": 2e-1,
        "p_init": 1e-1,
        "p1_error_model": {
            "X": 0.25,
            "Y": 0.25,
            "Z": 0.25,
            "L": 0.25,
        },
    },
)
engine = HybridEngine(error_model=error_model)
engine.use_seed(7)
results = engine.run(phir, shots=100)
print(sum(x == "11" for x in results["c"]))

May output 24, 25, 26, 27 etc.

[cqc-alec]

Using the state-vector backend

engine = HybridEngine(qsim="state-vector", error_model=error_model)

the results are still non-deterministic.

[jake-arkinstall]

I think the cause is

PECOS/python/pecos/engines/hybrid_engine.py

Line 174 in b16b27a

self.seed = self.use_seed(seed)

• engine.run has a default parameter initialize = True, and an optional parameter seed.
• If initialize is True, then set_seed is invoked with seed (which defaults to None)
• On seed=None, the seed is set to a random number.
• So set_seed prior to engine.run without initialize or seed parameters is effectively moot.

[cqc-alec]

I see, thanks. So if seed is provided we should always set initialize=True. What are the other effects of the initialize parameter?

[cqc-alec]

Note however that I am calling use_seed() explicitly, not passing it as an argument to run().

[cqc-alec]

... which means I actually need to set initialize=False, or do things a different way.

This behaviour must have changed recently.

[jake-arkinstall]

Yeah, that's what I mean. initialize=True is the default (and I think it's a good idea to leave it as the default as it calls
initialize_sim_components), which seems to mean that seed must be passed to run() instead of using set_seed.

It's possible that set_seed is only useful for subsequent runs on the same engine, but I don't know the codebase so I can't say for sure. I'd think that a PR on the above line 174 making set_seed only run if seed is not None would be suitable, but if set_seed is later set to have additional side effects that warrant it being invoked at least once, then tracking it being set inside set_seed and using that in the branching logic would be beneficial.

[cqc-alec]

I tried changing to:

# engine.use_seed(7)
results = engine.run(phir, shots=100, seed=7)

but still get non-deterministic results.

[jake-arkinstall]

For clarification, does the same behaviour apply to both sparse and statevector simulations when the seed is provided in engine.run?

I'm trying to track down the random seeding and uses of randomness. It doesn't look like the qulacs API supports seeding but I might be wrong. The sparse sim appears to use the python random number generation, so if it also isn't deterministic then somewhere along the line the random state is likely being overwritten.

[cqc-alec]

I've tried qsim="stabilizer" and qsim="state-vector"; both are non-deterministic.

[ciaranra]

Pablo mentioned previously that qulacs (as well as QuEST) doesn't support seeds as Jake mentioned in this thread. So not surprised by that behavior, although it should be made clear to users (at least probably give a warning if th seed is set and recommend the use of another backend). Stabilizer sims being non-deterministic is unexpected and disturbing...