BackendV3

[ElePT]

This RFC summarizes a proposal for BackendV3 that has been discussed for a few months already. The proposal is simple, but the discussions around the topic are not. The document focuses on adding historical context and capturing arguments for and against to clarify the motivation for the proposal.

[mtreinish]

Thanks for doing this @ElePT this is a great start. I did a quick read through it and left some initial thoughts inline.

[nonhermitian]

What is the benefit of having a Sampler and Estimator class here rather than just cutting them out entirely and just doing backend.sample(qc) and backend.estimate(qc)? If you go through the online Primitives examples the only thing they ever do is immediately call *.run(qc, parms) on the Sampler and Estimator; they are just middle man objects with no intrinsic value for the user.

[ElePT]

What is the benefit of having a Sampler and Estimator class here rather than just cutting them out entirely and just doing backend.sample(qc) and backend.estimate(qc)? If you go through the online Primitives examples the only thing they ever do is immediately call *.run(qc, parms) on the Sampler and Estimator; they are just middle man objects with no intrinsic value for the user.

From a developer and API perspective these middle man objects have a clear purpose and intrinsic value. They maintain the separation of concerns and deal with different server-side APIs. Primitives are not lightweight, and having a single backend-sampler-estimator would be chaotic and difficult to manage. The decoupling allows for better maintainability, stability, and less breaking changes overall. They also already exist and are well established and documented, and previous migrations have shown that drastic changes such as "wiping out" objects are increasingly difficult to carry out.

From a user perspective, the primitive examples also instantiate the primitives and define and set execution mode and options (shots, error mitigation and the works). If we were to merge backend and both primitives into a single backend-sampler-estimator, let's imagine that we are using qiskit-ibm-runtime, call service.backend("ibm_fez"), and we want to run an estimator in batch mode with 10k shots and a rep delay of 10^-4. What would be the user benefit of doing something like:

from qiskit_ibm_runtime import QiskitRuntimeService

service = QiskitRuntimeService()
backend = service.backend("ibm_fez")
backend.update_estimator_mode("batch")
backend.update_estimator_options({"shots":1024, "rep_delay": 1e-4})
job = backend.estimate([pub])

instead of the proposed

from qiskit_ibm_runtime import QiskitRuntimeService

service = QiskitRuntimeService()
backend = service.backend("ibm_fez")
estimator = backend.estimator(mode="batch", options={"shots":1024, "rep_delay": 1e-4})
job = estimator.run([pub])

?

Where the second one is compatible with current workflows and the first one would involve significant development and documentation (learning, etc) overhead.

[nonhermitian]

So the primary benefit is that I do not have to care/learn about another object. Especially one that is primarily an IBM Quantum construct. i.e. no one else has such things. Moreover, if you look at your example, you are treating the estimator instance as an extension of the backend rather than just using the backend itself. For example, the two options you set have nothing to do with computing expectation values, but are rather parameters used for execution on-chip, and have meaning outside of estimating. So something like:

service = QiskitRuntimeService()
backend = service.backend("ibm_fez")
backend.update_execution_mode("batch")
backend.update_execution_options({"shots":1024, "rep_delay": 1e-4})
job = backend.estimate([pub])

makes a ton more sense. Notice how I also changed the method names because nothing you have done is actually an estimator setting, but rather execution option. There are really 3 sets of options that exist today: execution, error suppression, and error mitigation. The sampler and estimator really just cherry pick from these sets. So tying all to the backend instance allows me to do:

job2 = backend.sample([qc])

and still have the above settings set. i.e. the job would be in batch mode with the selected shots and rep-delay.

I cannot respond to the separation of concerns part because I do not see where that arises here? Namely, the underlying mode calls and calls to backend.estimator and backend.sampler already mix the runtime and the backend object in your proposal.

[ElePT]

So the primary benefit is that I do not have to care/learn about another object. Especially one that is primarily an IBM Quantum construct. i.e. no one else has such things

My argument is not to introduce something new, but rather not disrupt what already exists. We are not building a new model from scratch, primitives already exist in our ecosystem. We are not picking between two equal options and my stance here is that removing them at this point would cause more harm than benefit. This is why, instead of removing the notion of primitives, this is a proposal to streamline their creation and smoothen the curve for users. With BackendV3 users don't need to pick a primitive, it's just something that is returned from the backend and users can run on demand. All without disrupting existing workflows.

[nonhermitian]

At the end of the day do what you will I guess. By abstracting away the execution mode, you are going in the right direction, but somehow are not willing to go the final step to backend nirvana. Just seeing the notion of estimator_options is disappointing as it indicates that people have not learned from previous experience with 14 different options classes, and think that runtime execution modes are independent constructs. As it stands now, there is little of value in this V3 proposal for end users to be happy about. Rather it seems to just be porting poor choices in the runtime interface over to Qiskit. I am happy though that at least backend.run is still there so those of us that try to write code that is provider agnostic still have a common point to work with (provided one just calls the sampler within the backend.run call for IBM HW)

[ElePT]

We are working with examples that are too overfitted to the IBMBackend when discussing something abstract and general. The BackendV3 interface doesn't limit how a downstream implementation can deal with things like "options", this is up to the specific implementation. Things like whether "rep_delay" is set at a backend level or primitive level (whether it's called estimator_option or backend_option or execution_option) is up to the implementer of the interface, all of them are possible in BackendV3. The only thing that the interface dictates is the sampler() factory instead of the sample() method. The base primitives already exist in Qiskit so we aren't porting anything from runtime, the design is provider agnostic. We are just unifying what exists in Qiskit.

[nonhermitian]

Yeah no your correct. That is a false statement on my end. The rest I stand behind though. E.g. there is no sane world where backend.run should not be paired with backend.sample as sampling and run are the same thing

[ihincks]

qiskit-ibm-runtime estimator currently has this construction signature:

    def __init__(
        self,
        mode: Optional[Union[BackendV2, Session, Batch]] = None,
        options: Optional[Union[Dict, EstimatorOptions]] = None,
    ):

[ihincks]

I have trouble finding the value of abstracting an execution mode whose implementation is optional, and when implemented, whose output is free-form. It seems like the real motivation for keeping .run in the abstraction is for historical continuity.

[ihincks]

Aliasing them would solve the problem of them being out of sync, but it doesn't seem to address the ambiguity problem, which I view as the real problem, as I suspect that the average user will not understand that it's an alias, and still wonder why the heck the same information seems to be in two places.

I would favor making a breaking change here.

[jyu00]

From Qiskit Runtime's perspective, our users just went through great pain to migrate from backend.run to the primitives. Personally I don't see enough value out of this proposal to force them to change their code yet again to go from Estimator(backend) to backend.estimator().

[ElePT]

From Qiskit Runtime's perspective, our users just went through great pain to migrate from backend.run to the primitives. Personally I don't see enough value out of this proposal to force them to change their code yet again to go from Estimator(backend) to backend.estimator().

Estimator(backend) and backend.estimator() are complementary, they can both be supported without forcing users to change their code. This is part of the backwards compatibility of the proposal. None of the proposed changes are breaking changes.

For what it's worth, I have shared the proposal with internal Runtime users and the response was quite positive. Choosing the right primitive (especially when switching between local simulation and HW execution) is one of the biggest pain points for users and the biggest user-facing motivation for this proposal.

[jyu00]

Estimator(backend) and backend.estimator() are complementary, they can both be supported without forcing users to change their code. This is part of the backwards compatibility of the proposal. None of the proposed changes are breaking changes.

If we don't plan to deprecate Estimator(backend), then it's hard to justify putting in the work to implement this new interface. Especially if we want to change all docs/tutorials etc to push the new interface. Having multiple ways of doing the same thing often raises questions and confusion.

Choosing the right primitive (especially when switching between local simulation and HW execution) is one of the biggest pain points for users and the biggest user-facing motivation for this proposal.

Qiskit Runtime has a local mode that's meant to allow users to switch between local simulation and IBM HW without changing the code besides the backend. It's not perfect, but wouldn't it be better to put the effort into improving it instead?