Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows: MSVC: Allocate wave_slice_dq array using mem_alloc_dq() #7491

Merged
merged 3 commits into from
Apr 1, 2023

Conversation

jhunkeler
Copy link
Collaborator

@jhunkeler jhunkeler commented Mar 13, 2023

Resolves N/A

Closes #6466, closes #6905

Related: #5578, spacetelescope/crds#790 (comment)

This replaces two variable length array declarations that are preventing Microsoft CL from succeeding. (Refer to #6466 above.)

I tested locally with:

  • Windows 10 Pro
  • MSVC 2022
  • Mambaforge
$ pytest -rsv jwst/cube_build/
============================= test session starts ==============================
platform linux -- Python 3.10.9, pytest-7.2.2, pluggy-1.0.0
crds_context: jwst_1064.pmap
rootdir: /home/jhunk/Downloads/omg/jwst, configfile: setup.cfg
plugins: asdf-2.14.3, requests-mock-1.10.0, openfiles-0.5.0, doctestplus-0.12.1, cov-4.0.0, ci-watson-0.6.1, jwst-1.9.5.dev37+g73135ae5b.d20230313
collected 16 items                                                             

jwst/cube_build/tests/test_configuration.py .....                        [ 31%]
jwst/cube_build/tests/test_cube_build_step.py .                          [ 37%]
jwst/cube_build/tests/test_miri_cubepars.py ...                          [ 56%]
jwst/cube_build/tests/test_nirspec_cubepars.py .                         [ 62%]
jwst/cube_build/tests/test_wcs.py ......                                 [100%]

============================== 16 passed in 3.48s ==============================

PS - I also noticed drizzle-1.13.7 fails to build on Windows, however at the time of this writing drizzle/master and drizzle/1.13.7 are the same... yet pip install git+https://github.com/spacetelescope/drizzle actually succeeds. Very strange. Maybe the archive is missing something?

Checklist for maintainers

  • added entry in CHANGES.rst within the relevant release section
  • updated or added relevant tests
  • updated relevant documentation
  • added relevant milestone
  • added relevant label(s)
  • ran regression tests, post a link to the Jenkins job below.
    How to run regression tests on a PR
  • Make sure the JIRA ticket is resolved properly

@jhunkeler jhunkeler changed the title Windwos: MSVC: Allocate wave_slice_dq using calloc+free Windows: MSVC: Allocate wave_slice_dq using calloc+free Mar 13, 2023
Copy link
Contributor

@pllim pllim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is also failing on master, so the failures here probably unrelated.

@jhunkeler jhunkeler force-pushed the msvc_cube_build_dq branch from 68f1550 to 3b3a76f Compare March 13, 2023 20:41
@jhunkeler jhunkeler changed the title Windows: MSVC: Allocate wave_slice_dq using calloc+free Windows: MSVC: Allocate wave_slice_dq array using mem_alloc_dq() Mar 13, 2023
@codecov
Copy link

codecov bot commented Mar 13, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (141ecbc) 77.56% compared to head (cc5555d) 77.56%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7491   +/-   ##
=======================================
  Coverage   77.56%   77.56%           
=======================================
  Files         452      452           
  Lines       36211    36211           
=======================================
  Hits        28086    28086           
  Misses       8125     8125           
Flag Coverage Δ *Carryforward flag
nightly 77.57% <ø> (ø) Carriedforward from 141ecbc

*This pull request uses carry forward flags. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jhunkeler jhunkeler force-pushed the msvc_cube_build_dq branch from 824ede4 to 867d495 Compare March 15, 2023 11:20
@jhunkeler jhunkeler force-pushed the msvc_cube_build_dq branch from 867d495 to b6e9f22 Compare March 26, 2023 00:24
@jemorrison
Copy link
Collaborator

I have a similar fix to some other cube_build updates on a branch I am working on and I have tested it quite a lot. This fix cleans up the remaining memory leaks.

@mcara
Copy link
Member

mcara commented Mar 31, 2023

I do not think the old code could have resulted in a memory leak but allocating large arrays on stack is not a good idea in general and allocating variable size arrays is not supported by all compilers (not a standard C feature depending on the standard version). For MS, in particular see https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-support-arriving-in-msvc/#variable-length-arrays

@jemorrison
Copy link
Collaborator

Running on a Mac M1 computer - the memory is smaller now than before the change to the dq arrays.

@pllim
Copy link
Contributor

pllim commented Mar 31, 2023

Just merge this already! 😸

@jemorrison
Copy link
Collaborator

I have a pr with exactly the same updates + other stuff. So if you merge this one I will rebase my code to grab you merge. It would helpful to merge this one soon because I want to submit my pr today and I can get it clean up with including this pr before I do that.

@pllim
Copy link
Contributor

pllim commented Mar 31, 2023

I don't have the power. Otherwise, I would have done it already. 😸

@hbushouse hbushouse added this to the Build 9.2 milestone Mar 31, 2023
cube_build
----------

- Windows: MSVC: Allocate ``wave_slice_dq`` array using ``mem_alloc_dq()`` [#7491]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, what does MSVC stand for?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment could be improved

@hbushouse hbushouse requested a review from a team as a code owner March 31, 2023 22:42
@hbushouse hbushouse merged commit ff692ab into spacetelescope:master Apr 1, 2023
@pllim
Copy link
Contributor

pllim commented Apr 1, 2023

Thanks!

@pllim
Copy link
Contributor

pllim commented Apr 20, 2023

@zacharyburnett , are you sure jwst won't work on Windows? Though hard to tell without actually running CI on it.

xref spacetelescope/hstcal#517

@zacharyburnett
Copy link
Collaborator

zacharyburnett commented Apr 20, 2023

@zacharyburnett , are you sure jwst won't work on Windows? Though hard to tell without actually running CI on it.

xref spacetelescope/hstcal#517

Oh nice, I didn't know about this! Shows what I know haha 😅

I can add some CI tests to the weekly cron to see how stable Windows support is

EDIT: made a PR #7562

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

JWST wheel fails to build >1.3.1 due to cube_build
6 participants