Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jlorincz/common testing #119

Merged
merged 66 commits into from
May 17, 2024
Merged

Jlorincz/common testing #119

merged 66 commits into from
May 17, 2024

Conversation

lajz
Copy link
Collaborator

@lajz lajz commented Apr 8, 2024

TODO:

  • Complete non-pytest testing helpers
  • move cli testing to use common infrastructure
  • figure out what assert statements should exist and where
  • fix up constants and verbosity settings
  • create better system for detecting tokamak
  • create the option to overwrite constant with environment variables
  • ability to run scripts from python, not pytest
  • xfailures and xsuccesses

@lajz lajz changed the base branch from main to develop April 8, 2024 08:15
@lajz

This comment was marked as duplicate.

@gtrevisan
Copy link
Member

gtrevisan commented Apr 10, 2024

in order to help us develop the parameter methods, we should use the xfail mark from pytest for methods that are known failures but should not prevent the whole workflow from succeeding.
https://docs.pytest.org/en/6.2.x/skipping.html#xfail-mark-test-functions-as-expected-to-fail

@gtrevisan

This comment was marked as outdated.

@gtrevisan

This comment was marked as outdated.

@lajz lajz requested a review from gtrevisan May 13, 2024 21:50
@gtrevisan
Copy link
Member

thank you! are you able to push the appropriate known failures list for each machine so that the tests pass? also, there might be some tweaks to be done on my earlier tests.

Copy link
Member

@gtrevisan gtrevisan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(1) as a final touch for now, please organize data in a more structured way, eg:

TEST_SHOTS = {
    "CMOD": <current CMOD_TEST_SHOTS>,
    "D3D: <current D3D_TEST_SHOTS>,
}

etc, it should be a lot easier to then read them up depending on the tokamak parameter.

(2) then, please remove the shot-by-shot info from the expected failures.

(3) finally, please fix the _fast logic which is currently flipped.

@gtrevisan gtrevisan marked this pull request as ready for review May 17, 2024 13:19
@gtrevisan gtrevisan merged commit 893a392 into develop May 17, 2024
5 checks passed
@gtrevisan gtrevisan deleted the jlorincz/common_testing branch May 17, 2024 13:27
@gtrevisan gtrevisan mentioned this pull request May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants