Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raise error for missing bounds in optimizer SNOBFIT #8638

Merged
merged 6 commits into from
Sep 14, 2022
Merged

Raise error for missing bounds in optimizer SNOBFIT #8638

merged 6 commits into from
Sep 14, 2022

Conversation

cpossel
Copy link
Contributor

@cpossel cpossel commented Aug 30, 2022

Summary

Raise error for invalid Nones in bounds (i.e. missing bounds) of optimizer SNOBFIT.
Closes #8580

Details and comments

Function signature is misleading, allowing None for bounds. The resulting error is non-intuitive. So, this PR adds raising a more concise error message. This includes also None values within the bounds list (can happen e.g. with ansatz UCCSD).

Raise error for invalid `None`s in bounds of optimizer SNOBFIT
@cpossel cpossel requested review from a team, manoelmarques and woodsp-ibm as code owners August 30, 2022 13:48
@qiskit-bot qiskit-bot added the Community PR PRs from contributors that are not 'members' of the Qiskit repo label Aug 30, 2022
@qiskit-bot
Copy link
Collaborator

Thank you for opening a new pull request.

Before your PR can be merged it will first need to pass continuous integration tests and be reviewed. Sometimes the review process can be slow, so please be patient.

While you're waiting, please feel free to review other open PRs. While only a subset of people are authorized to approve pull requests for merging, everyone is encouraged to review open pull requests. Doing reviews helps reduce the burden on the core team and helps make the project's code better for everyone.

One or more of the the following people are requested to review this:

@Cryoris
Copy link
Contributor

Cryoris commented Aug 31, 2022

Thanks for the contribution @cpossel! Could you add a test that checks this error is correctly raised? You could add it to
https://github.com/Qiskit/qiskit-terra/blob/main/test/python/algorithms/optimizers/test_optimizers.py

and check e.g. the following for an example of how to check that a code raises an error:
https://github.com/Qiskit/qiskit-terra/blob/2eee56616d50a9e26756f855ef4aa0135920ad78/test/python/algorithms/test_entangler_map.py#L51

@coveralls
Copy link

coveralls commented Aug 31, 2022

Pull Request Test Coverage Report for Build 3054844486

  • 2 of 2 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.0005%) to 84.376%

Totals Coverage Status
Change from base Build 3054338474: 0.0005%
Covered Lines: 58175
Relevant Lines: 68947

💛 - Coveralls

@cpossel
Copy link
Contributor Author

cpossel commented Aug 31, 2022

@Cryoris Test for failing SNOBFIT is added.
I didn't add any specific succeeding SNOBFIT test as it should already be captured (more or less) by the general scipy optimizer tests in the test suite.

@cpossel
Copy link
Contributor Author

cpossel commented Aug 31, 2022

I need to add the scikit-quant package to the requirements file in order to fix the error qiskit.exceptions.MissingOptionalLibraryError: "The 'scikit-quant' library is required to use 'SNOBFIT'. You can install it with 'pip install scikit-quant'." in the terra testing environment. Should it be added to the requirements-dev.txt file? Or is there another location to add this requirement to?

@woodsp-ibm
Copy link
Member

There is a unit test for the scikit-quant optimizers BOBYQA, SNOBFIT, IMFIL https://github.com/Qiskit/qiskit-terra/blob/main/test/python/algorithms/optimizers/test_optimizers_scikitquant.py but I don't think we have ever run it in CI. Any additional test should be in this file.

The documentation notes this on scikit-quant:

https://github.com/Qiskit/qiskit-terra/blob/a0db5817ac39222abeaef647a38df491d688d221/qiskit/algorithms/optimizers/__init__.py#L87-L92

@cpossel
Copy link
Contributor Author

cpossel commented Sep 2, 2022

The test cases in test_optimizers.py follow the logic to test the proper functioning of the optimizer itself while test_optimizers_scikitquant.py uses a holistic approach i.e. testing for the correct output of a VQE calculation with the chosen optimizer.
So, following this logic, the SNOBFIT test I want to add fits far better into test_optimizers.py. I'd also like to point out that the are already 2 scipy optimizer tests (unsing BFGS) in test_optimizers.py, namely test_scipy_optimizer and test_scipy_optimizer_callback.

test_optimizers_scikitquant.py uses the RealAmplitudes ansatz which sets the bounds by itself, so the initial error can't be reproduced with the test template in test_optimizers_scikitquant.py and multiple lines of code from test_optimizers.py would need to be copied to test_optimizers_scikitquant.py what seems very redundant and unintuitive for me.

Making the test case run without installing the third party package would still be possible catching the MissingOptionalLibraryError error the same way as done in test_optimizers_scikitquant.py

@woodsp-ibm
Copy link
Member

I am not sure I am fully following. The unit test method is just checking the bounds. All you need is to call minimize on it and pass some dummy objective function since it should fail immediately right. The run_optimizer is way more than is needed since that was done to check the optimizer could find the minimum of a function to check its correct working. Your test just checks that it fails as expected given an invalid bounds on the minimize right. To me it would be better to include that in the test suite for those 3. Yes it does use VQE - maybe finding some more straightforward function did not test the optimizers adequately. Ideally we would use as little other code as was needed to check their correctness - but I know sometimes things just don't work out.

@cpossel
Copy link
Contributor Author

cpossel commented Sep 5, 2022

Maybe my thoughts were a little too over-complicated... With your suggestions I added a minimal test function to test_scipy_optimizer_callback.py (ignoring the pattern of the other tests with VQE). I tested it locally and it showed the intended behavior so it should be fine now.

@cpossel
Copy link
Contributor Author

cpossel commented Sep 7, 2022

As far as I understand the output of the CI tests the test's failure is due to some CI configuration and not due to the code changes.
So, can anyone of you @Cryoris @woodsp-ibm @jakelishman look into the issue of the failed test, please?

@jakelishman
Copy link
Member

The tests are all passing now - I see that the last commit had a test failure that's since been re-run, and it was just caused by a dodgy network connection / overallocated hardware on Azure's side. Nothing to worry about - it's just transient and we don't have control over it.

@woodsp-ibm
Copy link
Member

Looks good, but can you add a release note for this bug fix.

@woodsp-ibm woodsp-ibm added the Changelog: Bugfix Include in the "Fixed" section of the changelog label Sep 12, 2022
@woodsp-ibm woodsp-ibm added this to the 0.22 milestone Sep 12, 2022
@cpossel
Copy link
Contributor Author

cpossel commented Sep 14, 2022

@woodsp-ibm Okay, I added the release note. Is anything else missing?

Copy link
Member

@woodsp-ibm woodsp-ibm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. All looks good to me now.

@mergify mergify bot merged commit 3cc9c17 into Qiskit:main Sep 14, 2022
ElePT pushed a commit to ElePT/qiskit that referenced this pull request Jun 27, 2023
* Update snobfit.py

Raise error for invalid `None`s in bounds of optimizer SNOBFIT

* Add SNOBFIT test for missing bounds

* Move refactored SNOBFIT test for missing bounds to test_optimizers_scikitquant.py

* Remove obsolete SNOBFIT import

* Add release note

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ElePT pushed a commit to ElePT/qiskit-algorithms-test that referenced this pull request Jul 17, 2023
* Update snobfit.py

Raise error for invalid `None`s in bounds of optimizer SNOBFIT

* Add SNOBFIT test for missing bounds

* Move refactored SNOBFIT test for missing bounds to test_optimizers_scikitquant.py

* Remove obsolete SNOBFIT import

* Add release note

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changelog: Bugfix Include in the "Fixed" section of the changelog Community PR PRs from contributors that are not 'members' of the Qiskit repo mod: algorithms Related to the Algorithms module
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Bug using SNOBFIT in combination with UCCSD (VQE) / misleading function signature of optimizer SNOBFIT
6 participants