Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace opencv-python with pre-built CPU-only opencv-python-headless #7570

Merged

Conversation

zacharyburnett
Copy link
Collaborator

@zacharyburnett zacharyburnett commented Apr 26, 2023

Closes spacetelescope/stenv#103

This PR tests replacing opencv-python with opencv-python-headless, a pre-built CPU-only binary, to resolve the issue raised in spacetelescope/stenv#103

Checklist for maintainers

  • added entry in CHANGES.rst within the relevant release section
  • updated or added relevant tests
  • updated relevant documentation
  • added relevant milestone
  • added relevant label(s)
  • ran regression tests, post a link to the Jenkins job below.
    How to run regression tests on a PR
  • Make sure the JIRA ticket is resolved properly

@codecov
Copy link

codecov bot commented Apr 27, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (5fdad6f) 77.56% compared to head (d12b5f8) 77.56%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7570   +/-   ##
=======================================
  Coverage   77.56%   77.56%           
=======================================
  Files         455      455           
  Lines       36340    36340           
=======================================
  Hits        28188    28188           
  Misses       8152     8152           
Flag Coverage Δ *Carryforward flag
nightly 77.62% <ø> (ø) Carriedforward from 5fdad6f

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@zacharyburnett zacharyburnett marked this pull request as ready for review April 27, 2023 14:20
@zacharyburnett zacharyburnett requested a review from a team as a code owner April 27, 2023 14:20
@zacharyburnett zacharyburnett changed the title attempt to use pre-built binary of opencv replace opencv-python with pre-built CPU-only opencv-python-headless Apr 27, 2023
@zacharyburnett
Copy link
Collaborator Author

upstream tests at spacetelescope/stcal#164

@hbushouse
Copy link
Collaborator

Looks like the regtest run was aborted. Will there be another?

@zacharyburnett
Copy link
Collaborator Author

zacharyburnett commented Apr 27, 2023

Looks like the regtest run was aborted. Will there be another?

yes I ran one at https://plwishmaster.stsci.edu:8081/blue/organizations/jenkins/RT%2FJWST-Developers-Pull-Requests/detail/JWST-Developers-Pull-Requests/697/pipeline

It looks like there were quite a few failures:
https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/697/testReport/
with quite a few of them having the following:

../miniconda/envs/tmp_env0/lib/python3.9/site-packages/stdatamodels/jwst/transforms/converters/jwst_models.py:34: in to_yaml_tree_transform
    xll = [list(m) for m in model.xmodels]
E   TypeError: 'Polynomial1D' object is not iterable

@braingram
Copy link
Collaborator

Looks like the regtest run was aborted. Will there be another?

yes I ran one at https://plwishmaster.stsci.edu:8081/blue/organizations/jenkins/RT%2FJWST-Developers-Pull-Requests/detail/JWST-Developers-Pull-Requests/697/pipeline

It looks like there were quite a few failures: https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/697/testReport/ with quite a few of them having the following:

../miniconda/envs/tmp_env0/lib/python3.9/site-packages/stdatamodels/jwst/transforms/converters/jwst_models.py:34: in to_yaml_tree_transform
    xll = [list(m) for m in model.xmodels]
E   TypeError: 'Polynomial1D' object is not iterable

Those errors also appeared in CI runs for this PR for stdatamodels (where @nden mentioned they should be resolved by new reference files).

spacetelescope/stdatamodels#160 (comment)

and are likely related to: spacetelescope/stdatamodels#124

@hbushouse
Copy link
Collaborator

That's because you're letting stdatamodels float to the latest. Must pin to "stdatamodels==1.4.0".

@zacharyburnett
Copy link
Collaborator Author

That's because you're letting stdatamodels float to the latest. Must pin to "stdatamodels==1.4.0".

Ah right, I remember your announcement

@zacharyburnett
Copy link
Collaborator Author

@zacharyburnett
Copy link
Collaborator Author

regression test is all green now 🎉

@hbushouse
Copy link
Collaborator

regression test is all green now 🎉

I see that. One caveat though: the regtests only run on Linux. How do we know everything's OK with builds on a Mac?

@zacharyburnett
Copy link
Collaborator Author

Good point, I started a dev job at https://plwishmaster.stsci.edu:8081/blue/organizations/jenkins/RT%2FJWST-devdeps/detail/JWST-devdeps/656/pipeline/ as well with stdatamodels==1.4.0 to cover the MacOS tests

@zacharyburnett
Copy link
Collaborator Author

The MacOS tests are running into the ephemeral socket errors: https://plwishmaster.stsci.edu:8081/blue/organizations/jenkins/RT%2FJWST-devdeps/detail/JWST-devdeps/656/pipeline/194#step-196-log-29

However, it looks like there were no import errors (so far), so we'll see if the test failures are just socket errors and precision differences between platforms

@zacharyburnett
Copy link
Collaborator Author

@zacharyburnett zacharyburnett merged commit aa6e9ef into spacetelescope:master Apr 28, 2023
@zacharyburnett zacharyburnett deleted the opencv_headless branch April 28, 2023 17:22
@hbushouse hbushouse added this to the Build 9.3 milestone Apr 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"conda env create" crashes for MacOS + python 3.10, 3.11
3 participants