Optimization benchmarking · Independent lesson
An optimization benchmark starts with the instance and the stopping rule
An independent QubitWire lesson on this topic, based on the source reading below. It is not a transcript or a summary of the recording.
An optimization method tries to find a good feasible answer under a defined objective. To compare methods, first specify the instances they must solve, the constraints a valid answer must satisfy and the quality threshold that counts as success. A runtime number without those choices can hide a much easier task.
QOBLIB is the benchmarking resource linked in the official interview description. Use the library and its associated paper to inspect the problem families, instance definitions and comparison procedure before repeating a performance headline. The editorial opportunity is to make those choices legible to readers who care about applications but do not work on solvers.
Quality and time are related. One method may return a modest answer quickly while another improves steadily over a longer run. A useful comparison can ask how long each takes to reach the same target, or compare the quality reached under the same time budget. Both approaches need repeated trials if the method is stochastic.
The total budget should include necessary preparation and tuning. A quantum routine can be one component of a larger solver, so the classical work around it belongs in any practical assessment.
Questions to work through
Why test more than one favorable instance?
Different structures can favor different methods. A representative, disclosed instance set helps distinguish a robust improvement from a carefully chosen example.
How should infeasible answers be treated?
They should not be scored as successful solutions merely because their objective value looks attractive. Constraint satisfaction is part of the task.
What does time to target add?
It compares methods at a shared quality threshold. The target, failure handling and probability of reaching it must be stated.
Source reading
- youtube.com ↗
https://www.youtube.com/watch?v=D56C9YVNcZY
- zib-aopt.github.io ↗
https://zib-aopt.github.io/QOBLIB/