Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v0.5 #117

Merged
merged 284 commits into from
May 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
284 commits
Select commit Hold shift + click to select a range
0d6018b
Fix type bug
lajz Mar 6, 2024
cc90ec3
Minor parameter method bug fixes
lajz Mar 6, 2024
0fa29f5
Fix typo in open_tree call
lajz Mar 6, 2024
d63ab76
Fix None args bug
lajz Mar 6, 2024
b36e64b
Fix support issues for python 3.7
Mar 15, 2024
443d9b0
Add support for d3d database
lajz Mar 15, 2024
466aa60
Add d3d basic example
lajz Mar 15, 2024
04fad06
Fix driver for d3d sql database
lajz Mar 15, 2024
9d9ce88
Fix db name
lajz Mar 15, 2024
cc00996
Fixed d3d sql connection
lajz Mar 18, 2024
1989e89
Get d3d testing basics working
lajz Mar 20, 2024
93fcf06
Fix time rounding errors
lajz Mar 20, 2024
957f427
Remove references to user in tests
lajz Mar 20, 2024
4850219
add convenience methods for data and dims, include casting
gtrevisan Mar 22, 2024
a07046a
cast only data by default
gtrevisan Mar 22, 2024
2322ecb
refactor other calls
gtrevisan Mar 22, 2024
f4a01f9
fixup! cast only data by default
gtrevisan Mar 22, 2024
f5c06bd
spawn min num of processes
gtrevisan Mar 22, 2024
eece90c
revamp db login parsing
gtrevisan Mar 22, 2024
cea402c
allow arguments in get_data
gtrevisan Mar 25, 2024
52f5293
add script for simple queries
gtrevisan Mar 26, 2024
29d0fb7
simplify init methods
gtrevisan Mar 26, 2024
29bbb21
fix warning of missing self
gtrevisan Mar 26, 2024
a498b0f
fix port for CMOD database
gtrevisan Mar 26, 2024
d371f38
fix non-standard port for ODBC
gtrevisan Mar 26, 2024
566915b
implement driver fallback, remove curly braces
gtrevisan Mar 26, 2024
816f89e
rename to sql, interactive only on debug
gtrevisan Mar 26, 2024
8d85162
add script to test mdsplus
gtrevisan Mar 26, 2024
ea2dfa0
add current truth values
gtrevisan Mar 26, 2024
9868d63
delay import of netCDF4
gtrevisan Mar 28, 2024
5cfac04
add quick efit workflow
gtrevisan Mar 28, 2024
093afd8
add quick test for sql, mdsplus, and efit
gtrevisan Mar 28, 2024
334b18c
remove old files
gtrevisan Mar 28, 2024
d967d34
rebase deps from develop
gtrevisan Mar 28, 2024
75605dc
rebase gitignore from develop
gtrevisan Mar 28, 2024
916fb94
minor changes to d3d and cmod test, remove init from test
gtrevisan Mar 28, 2024
0878382
remove test data folder
gtrevisan Mar 28, 2024
9a35506
rename test to tests
gtrevisan Mar 28, 2024
c347545
Merge pull request #112 from MIT-PSFC/fix-get-data
gtrevisan Mar 29, 2024
11d48b9
bump dev version
gtrevisan Mar 29, 2024
b9ceca8
Merge branch 'develop' into jlorincz/d3d_thin_support
lajz Mar 29, 2024
9523145
Merge branch 'develop' of github.com:MIT-PSFC/disruption-py into jlor…
lajz Mar 29, 2024
59e66cb
Fixes for renamed test to tests
lajz Mar 29, 2024
66c2582
catch shorten_path errors
gtrevisan Apr 2, 2024
26ddf6b
replace get_record_data with get_data_with_dims
gtrevisan Apr 2, 2024
2dd3f6f
gather data and astype calls
gtrevisan Apr 2, 2024
94f4dfb
Merge pull request #68 from MIT-PSFC/jlorincz/d3d_thin_support
gtrevisan Apr 2, 2024
7d422d9
New generalized testing structure
lajz Apr 8, 2024
5e2094c
rename workflow to Docs
gtrevisan Apr 8, 2024
f09916e
add env to control tests
gtrevisan Apr 8, 2024
a1a8c28
avoid FQDNs for DIII-D
gtrevisan Apr 8, 2024
713ad29
add draft of pytest workflow
gtrevisan Apr 8, 2024
c170a3b
add draft of Lint workflow
gtrevisan Apr 8, 2024
27a0796
Merge pull request #120 from MIT-PSFC/tests-lint
gtrevisan Apr 8, 2024
cff8aae
Improved generalization across tokamaks
lajz Apr 9, 2024
839072a
restric lint to main and dev branches (#122)
gtrevisan Apr 10, 2024
a55fbcd
restrict to main and dev branches, add manual trigger
gtrevisan Apr 10, 2024
0c1cf72
remove netcdf dep
gtrevisan Apr 10, 2024
4926b56
lock deps
gtrevisan Apr 10, 2024
cab6388
Merge pull request #123 from MIT-PSFC/h5netcdf
gtrevisan Apr 10, 2024
9bc4d78
remove old pip files (#124)
gtrevisan Apr 10, 2024
7142382
fixup! restrict to main and dev branches, add manual trigger
gtrevisan Apr 10, 2024
4d9d1b5
minor yaml and debug changes
gtrevisan Apr 17, 2024
341a1f8
setup tunnel earlier
gtrevisan Apr 17, 2024
96c9e64
Test against sql running
lajz Apr 17, 2024
6ada81f
Merge pull request #121 from MIT-PSFC/tests-pytest
gtrevisan Apr 17, 2024
483df9a
use NFS folders to get tokamak
gtrevisan Apr 17, 2024
ec98531
skip workflows on draft PRs (#125)
gtrevisan Apr 17, 2024
15704ad
add Dependabot workflow
gtrevisan Apr 17, 2024
1e6d82b
fix typo from f- to r-string
gtrevisan Apr 19, 2024
3a464ab
Clean up test against sql when run without pytest
lajz Apr 21, 2024
76e3d3d
Fix to add more clarity around expected failures
lajz Apr 21, 2024
c7cdb2f
drop importlib backport for py37 (#126)
gtrevisan Apr 22, 2024
5b9aedf
order keys in 1st section, update info
gtrevisan Apr 22, 2024
c4f2ef1
drop py38 and old deps, update minor, zero-out patch
gtrevisan Apr 22, 2024
24f9106
Better support for missing data and cli
lajz Apr 22, 2024
47099fd
Improve cli experience
lajz Apr 22, 2024
f040185
add ipykernel to lab deps
gtrevisan Apr 22, 2024
14721f8
lock deps
gtrevisan Apr 22, 2024
6d925a8
Merge pull request #127 from MIT-PSFC/revamp-deps
gtrevisan Apr 22, 2024
dce1a5f
allow tweaking tokamak from env vars
gtrevisan Apr 22, 2024
f0c027c
add Install workflow
gtrevisan Apr 22, 2024
4f893c6
rerun poetry lock rather than just checking
gtrevisan Apr 22, 2024
0425e4e
add hdf5 extras for pandas
gtrevisan Apr 22, 2024
ef49041
lock deps
gtrevisan Apr 22, 2024
7f31e4c
Merge pull request #131 from MIT-PSFC/fix-deps
gtrevisan Apr 22, 2024
db47024
tweak feature test
gtrevisan Apr 19, 2024
33a3681
skip if on DIII-D
gtrevisan Apr 19, 2024
a32f87f
add to tests if on C-MOD
gtrevisan Apr 19, 2024
3f8e94c
revamp feature test
gtrevisan Apr 22, 2024
1513382
slim down fast tests even further
gtrevisan Apr 22, 2024
9205839
explicit duration of tests
gtrevisan Apr 22, 2024
ccdb166
single-shot test for GH actions
gtrevisan Apr 22, 2024
250a326
print versions and packages
gtrevisan Apr 22, 2024
3978590
silent curl
gtrevisan Apr 22, 2024
3d9f309
update mkdocs
gtrevisan Apr 22, 2024
d9b8fa8
lock deps
gtrevisan Apr 22, 2024
985d090
Merge pull request #132 from MIT-PSFC/update-deps
gtrevisan Apr 22, 2024
982ec0b
Avoid errors for missing data
lajz Apr 24, 2024
b2f0ed4
Merge branch 'jlorincz/common_testing' of github.com:MIT-PSFC/disrupt…
lajz Apr 24, 2024
a6f44de
Merge pull request #129 from MIT-PSFC/tests-deps
gtrevisan Apr 24, 2024
d80d00f
Merge pull request #128 from MIT-PSFC/tests-features
gtrevisan Apr 24, 2024
185629f
fix FutureWarning from to_hdf
gtrevisan Apr 24, 2024
d96e429
Merge pull request #130 from MIT-PSFC/tests-install
gtrevisan Apr 24, 2024
4ca499d
New generalized testing structure
lajz Apr 8, 2024
dcf0159
Improved generalization across tokamaks
lajz Apr 9, 2024
1cd6025
Test against sql running
lajz Apr 17, 2024
ea02632
use NFS folders to get tokamak
gtrevisan Apr 17, 2024
c034568
Clean up test against sql when run without pytest
lajz Apr 21, 2024
80b2d16
Fix to add more clarity around expected failures
lajz Apr 21, 2024
d354c9d
Better support for missing data and cli
lajz Apr 22, 2024
6e94244
Improve cli experience
lajz Apr 22, 2024
aec6e35
Avoid errors for missing data
lajz Apr 24, 2024
cd7f0eb
allow tweaking tokamak from env vars
gtrevisan Apr 22, 2024
14cefc6
Merge branch 'jlorincz/common_testing' of github.com:MIT-PSFC/disrupt…
lajz Apr 24, 2024
9034c16
New generalized testing structure
lajz Apr 8, 2024
f3c4c75
Improved generalization across tokamaks
lajz Apr 9, 2024
0913642
Test against sql running
lajz Apr 17, 2024
a9759f7
use NFS folders to get tokamak
gtrevisan Apr 17, 2024
2a1e45c
Clean up test against sql when run without pytest
lajz Apr 21, 2024
c0089ec
Fix to add more clarity around expected failures
lajz Apr 21, 2024
bf3c772
Better support for missing data and cli
lajz Apr 22, 2024
510246b
Improve cli experience
lajz Apr 22, 2024
ce5d9f5
Avoid errors for missing data
lajz Apr 24, 2024
6461563
allow tweaking tokamak from env vars
gtrevisan Apr 22, 2024
4f2523c
New generalized testing structure
lajz Apr 8, 2024
6799f76
Improved generalization across tokamaks
lajz Apr 9, 2024
d5f0b09
Merge branch 'jlorincz/common_testing' of github.com:MIT-PSFC/disrupt…
lajz Apr 24, 2024
2eef615
Remove single tokamak testing as replaced by generalized testing
lajz Apr 24, 2024
af9ff85
Clean up report string
lajz Apr 24, 2024
67671cb
fixed zerror bugs in basic_parameter_methods.py and added debug_zcur.…
AlexSaperstein Apr 29, 2024
f8d1f43
cleaned up debug script output
AlexSaperstein Apr 29, 2024
8bec590
removed my comments from basic_parameter_methods.py
AlexSaperstein Apr 29, 2024
f163fa6
removed debug_zcur.py from repo. Removed comments and other extraneou…
AlexSaperstein Apr 29, 2024
b545f76
remove debug comment
gtrevisan May 1, 2024
d0ae70c
Merge pull request #134 from MIT-PSFC/zcur_fix
gtrevisan May 1, 2024
8fcd771
sort jobs in Lint workflow
gtrevisan May 1, 2024
fffb227
add pylint job
gtrevisan May 1, 2024
808ce40
add black job
gtrevisan May 1, 2024
b8bfb86
install yamllint from lockfile
gtrevisan May 1, 2024
79652ad
Fix common testing for d3d
lajz May 2, 2024
a621f5d
add z parameters to xfailures
gtrevisan May 3, 2024
156a9be
add Te width to xfailures
gtrevisan May 3, 2024
8bed3e4
Fix pytest issues
lajz May 3, 2024
df2c6f6
Restructure for eval files for improved clarity
lajz May 3, 2024
5800674
Merge pull request #135 from MIT-PSFC/tests-pylint
gtrevisan May 3, 2024
72196df
Fix test naming
lajz May 8, 2024
28bc286
Change default python test running to fail slow
lajz May 8, 2024
e4a6e5d
Proposed zcur fix after back-and-forth with Bob. apply -1x to z_error…
AlexSaperstein May 9, 2024
c6c3dc6
simplified zcur calc correction
AlexSaperstein May 9, 2024
65d8a8c
Merge pull request #136 from MIT-PSFC/alex_zcur_fix_2
gtrevisan May 10, 2024
bc03937
Improve interoperability of feature testing
lajz May 10, 2024
db8c17d
d3d fixes for interoperable feature testing
lajz May 11, 2024
9ac663b
Better system for shot id selection
lajz May 13, 2024
abf2914
Fix shot id issue
lajz May 13, 2024
1f11d8c
Add shot info to expected failures
lajz May 14, 2024
4316f93
Fix xfail mark to continue execution and better monkey patch
lajz May 14, 2024
ceac5fd
Fix cast runtime warning bug and add xfail shot granularity
lajz May 14, 2024
6775998
Fix incorrect constants
lajz May 14, 2024
cdd5d4f
remove quick example, superseded by efit example
gtrevisan May 14, 2024
48d8d2c
check env var before folders
gtrevisan May 14, 2024
0a869ac
use common routines for tokamak and handler
gtrevisan May 14, 2024
550b132
Value rather than Runtime errors
gtrevisan May 14, 2024
05d38e1
remove old testing script
gtrevisan May 14, 2024
d3a1e52
run all tests in workflow
gtrevisan May 14, 2024
cc20a14
Widen usage of tokamak mapping infrastructure
lajz May 14, 2024
1f7fcfb
Improve mapping method naming
lajz May 14, 2024
de8a358
Resolve merge conflicts
lajz May 14, 2024
81bea4b
fix simple tests
gtrevisan May 15, 2024
42a3c2b
Remove expected shot failures
lajz May 15, 2024
d70c7ae
Single dictionaries for test constants
lajz May 15, 2024
6582012
fix logic for fast tests
gtrevisan May 17, 2024
825be59
remove arg set to default
gtrevisan May 17, 2024
893a392
Merge pull request #119 from MIT-PSFC/jlorincz/common_testing
gtrevisan May 17, 2024
4113d79
black! disruption_py/cli/cli.py
gtrevisan May 17, 2024
ae78229
black! disruption_py/cli/evaluate_methods.py
gtrevisan May 17, 2024
01e54f5
black! disruption_py/cli/generate_datasets.py
gtrevisan May 17, 2024
ecd3569
black! disruption_py/cli/setup_script.py
gtrevisan May 17, 2024
9ea75fa
black! disruption_py/databases/d3d_database.py
gtrevisan May 17, 2024
8684214
black! disruption_py/databases/database.py
gtrevisan May 17, 2024
2721f7a
black! disruption_py/databases/dummy_database.py
gtrevisan May 17, 2024
7051d8a
black! disruption_py/databases/__init__.py
gtrevisan May 17, 2024
6e96b3f
black! disruption_py/handlers/cmod_handler.py
gtrevisan May 17, 2024
c852ec6
black! disruption_py/handlers/d3d_handler.py
gtrevisan May 17, 2024
1ca5fc8
black! disruption_py/handlers/handler.py
gtrevisan May 17, 2024
cd6b7a3
black! disruption_py/handlers/__init__.py
gtrevisan May 17, 2024
263f549
black! disruption_py/handlers/multiprocessing_helper.py
gtrevisan May 17, 2024
58bf300
black! disruption_py/__init__.py
gtrevisan May 17, 2024
8152e3a
black! disruption_py/mdsplus_integration/mds_connection.py
gtrevisan May 17, 2024
fa227db
black! disruption_py/settings/enum_options.py
gtrevisan May 17, 2024
bc0695d
black! disruption_py/settings/existing_data_request.py
gtrevisan May 17, 2024
a0e6144
black! disruption_py/settings/__init__.py
gtrevisan May 17, 2024
fda1341
black! disruption_py/settings/log_settings.py
gtrevisan May 17, 2024
2fab22e
black! disruption_py/settings/output_type_request.py
gtrevisan May 17, 2024
4551816
black! disruption_py/settings/set_times_request.py
gtrevisan May 17, 2024
1dc78a8
black! disruption_py/settings/shot_data_request.py
gtrevisan May 17, 2024
14051f6
black! disruption_py/settings/shot_ids_request.py
gtrevisan May 17, 2024
a200a1a
black! disruption_py/settings/shot_settings.py
gtrevisan May 17, 2024
3fabd58
black! disruption_py/shots/cmod_shot_manager.py
gtrevisan May 17, 2024
6825f49
black! disruption_py/shots/d3d_shot_manager.py
gtrevisan May 17, 2024
f2b5275
black! disruption_py/shots/helpers/cached_method_props.py
gtrevisan May 17, 2024
b525e21
black! disruption_py/shots/helpers/method_caching.py
gtrevisan May 17, 2024
1d6049d
black! disruption_py/shots/helpers/method_optimizer.py
gtrevisan May 17, 2024
e57df17
black! disruption_py/shots/helpers/populate_shot.py
gtrevisan May 17, 2024
e7551d6
black! disruption_py/shots/parameter_methods/cmod/basic_parameter_met…
gtrevisan May 17, 2024
af0cbb3
black! disruption_py/shots/parameter_methods/cmod/built_in.py
gtrevisan May 17, 2024
53d8016
black! disruption_py/shots/parameter_methods/d3d/basic_parameter_meth…
gtrevisan May 17, 2024
169db8e
black! disruption_py/shots/parameter_methods/d3d/built_in.py
gtrevisan May 17, 2024
4d80861
black! disruption_py/shots/parameter_methods/d3d/efit_parameter_metho…
gtrevisan May 17, 2024
d129d28
black! disruption_py/shots/shot_manager.py
gtrevisan May 17, 2024
2b19894
black! disruption_py/shots/shot_props.py
gtrevisan May 17, 2024
74355ef
black! disruption_py/utils/command_utils.py
gtrevisan May 17, 2024
c067250
black! disruption_py/utils/constants.py
gtrevisan May 17, 2024
e47de21
black! disruption_py/utils/environment_vars.py
gtrevisan May 17, 2024
88c850c
black! disruption_py/utils/eval/data_difference.py
gtrevisan May 17, 2024
c2df0dd
black! disruption_py/utils/eval/eval_against_sql.py
gtrevisan May 17, 2024
191eed3
black! disruption_py/utils/mappings/mappings_helpers.py
gtrevisan May 17, 2024
08f8400
black! disruption_py/utils/mappings/tokamak_helpers.py
gtrevisan May 17, 2024
1c82ee8
black! disruption_py/utils/mappings/tokamak.py
gtrevisan May 17, 2024
e332200
black! disruption_py/utils/math_utils.py
gtrevisan May 17, 2024
49efa20
black! disruption_py/utils/ml/preprocessing.py
gtrevisan May 17, 2024
b6c2efe
black! disruption_py/utils/utils.py
gtrevisan May 17, 2024
1c65f8c
black! examples/all_defaults_example.py
gtrevisan May 17, 2024
8b5bd40
black! examples/basic_example_1.py
gtrevisan May 17, 2024
e650e30
black! examples/basic_example_2.py
gtrevisan May 17, 2024
fe3b073
black! examples/custom_shot_id_request.py
gtrevisan May 17, 2024
75316fe
black! examples/database_example.py
gtrevisan May 17, 2024
4565312
black! examples/decorator_functions_docs.py
gtrevisan May 17, 2024
b1e6f34
black! examples/no_database.py
gtrevisan May 17, 2024
fb15b9b
black! examples/prad_example.py
gtrevisan May 17, 2024
75dc011
black! examples/shot_data_request_docs.py
gtrevisan May 17, 2024
6b787aa
black! scripts/archive/eval.py
gtrevisan May 17, 2024
aa4dab1
black! scripts/archive/examples/d3d_example.py
gtrevisan May 17, 2024
63caca8
black! scripts/archive/examples/dl_example.py
gtrevisan May 17, 2024
3d2c5a9
black! scripts/archive/lyman_data.py
gtrevisan May 17, 2024
5f4febe
black! scripts/archive/profile_shots.py
gtrevisan May 17, 2024
1529056
black! scripts/archive/summarize_database.py
gtrevisan May 17, 2024
2a6260d
black! scripts/archive/train.py
gtrevisan May 17, 2024
f80ff62
black! scripts/archive/TS_profiles.py
gtrevisan May 17, 2024
5827aeb
black! scripts/archive/validate_shots.py
gtrevisan May 17, 2024
df6946b
black! scripts/archive/visualize_workflow.py
gtrevisan May 17, 2024
515d4a1
black! scripts/evaluate_data.py
gtrevisan May 17, 2024
7d2f230
black! scripts/generate_datasets.py
gtrevisan May 17, 2024
89a6af4
black! tests/conftest.py
gtrevisan May 17, 2024
adbfe5b
black! tests/test_against_sql.py
gtrevisan May 17, 2024
96c17c5
black! tests/test_features.py
gtrevisan May 17, 2024
ab29438
Merge pull request #138 from MIT-PSFC/apply-black
gtrevisan May 17, 2024
7b22f00
Merge pull request #139 from MIT-PSFC/tests-black
gtrevisan May 17, 2024
7526175
update README
gtrevisan May 17, 2024
a5ef7d9
set RC by poetry check
gtrevisan May 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---

version: 2
updates:

- package-ecosystem: "pip"
directory: "/"
versioning-strategy: increase-if-necessary
schedule:
interval: "monthly"
reviewers:
- gtrevisan

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
reviewers:
- gtrevisan
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---

# Simple workflow for deploying static content to GitHub Pages
name: Deploy documentation to Pages
name: Docs

# yamllint disable-line rule:truthy
on:
Expand Down
69 changes: 69 additions & 0 deletions .github/workflows/install.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---

name: Install

# yamllint disable-line rule:truthy
on:
push:
branches:
- 'main*'
- 'dev*'
paths:
- "pyproject.toml"
- "poetry.lock"
- ".pip/*"
pull_request:
branches:
- 'main*'
- 'dev*'
paths:
- "pyproject.toml"
- "poetry.lock"
- ".pip/*"


jobs:


poetry:
runs-on: ubuntu-latest
if: |
github.event_name == 'push' ||
github.event.pull_request.draft == false
steps:

- name: Checkout
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.10'

- name: Install Poetry
uses: snok/install-poetry@v1

- name: Print versions
run: |
python --version
poetry --version

- name: Check lock files
run: |
poetry lock --no-update
bash .pip/lock.sh
git diff --exit-code || true
poetry check

- name: Poetry install
run: poetry install

- name: List packages
run: |
poetry run pip list \
| tee pip.log
grep disruption.py pip.log

- name: Test import
run: |
poetry run python -c "import disruption_py"
130 changes: 130 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
---

name: Lint

# yamllint disable-line rule:truthy
on:
push:
branches:
- 'main*'
- 'dev*'
pull_request:
branches:
- 'main*'
- 'dev*'

jobs:


black:
runs-on: ubuntu-latest
if: |
github.event_name == 'push' ||
github.event.pull_request.draft == false
steps:

- uses: actions/checkout@v4

- name: Install black
run: |
grep -oP 'black.*?\ ' .pip/dev.txt \
| xargs pip install

- name: Run black
run: |
black --version
black --check .


pylint:
runs-on: ubuntu-latest
if: |
github.event_name == 'push' ||
github.event.pull_request.draft == false
steps:

- uses: actions/checkout@v4

- name: Install requirements
run: |
pip install -r .pip/main.txt -r .pip/dev.txt -r .pip/lab.txt
pip install -e .

- name: Run pylint
run: |
pylint --version
pylint disruption_py examples tests \
--msg-template='{path}:{line}:{column}: |{msg_id} {symbol}| {msg}' \
| tee pylint.log \
| grep -F '*****'

- name: Compute statistics
run: |
cut -s -d '|' -f 2 pylint.log \
| sort \
| uniq -c \
| sort -n \
| tee stats.log

- name: Extract worst offender
run: |
WORST=$(tail -n1 stats.log | grep -o '[IRCWEF][0-9]*')
grep "|$WORST" pylint.log

- name: Print final summary
run: |
tail pylint.log \
| grep '^Your code has been rated'


shellcheck:
runs-on: ubuntu-latest
if: |
github.event_name == 'push' ||
github.event.pull_request.draft == false
steps:

- uses: actions/checkout@v4

- name: Install shellcheck
run: sudo apt-get install shellcheck

- name: Run shellcheck
run: |
shellcheck --version
find -type f -not -path '*/.git/*' \
| xargs grep -l '^#!/bin/bash' \
| while read -r FILE
do
echo "--> $FILE"
shellcheck -x "$FILE"
done


yamllint:
runs-on: ubuntu-latest
if: |
github.event_name == 'push' ||
github.event.pull_request.draft == false
steps:

- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: '3.10'

- name: Install yamllint
run: |
grep -oP 'yamllint.*?\ ' .pip/dev.txt \
| xargs pip install

- name: Run yamllint
run: |
yamllint --version
find -type f -iname '*.yml' -or -iname '*.yaml' \
| while read -r FILE
do
echo "--> $FILE"
yamllint "$FILE"
done
148 changes: 148 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
---

name: Tests

# yamllint disable-line rule:truthy
on:
push:
branches:
- 'main*'
- 'dev*'
pull_request:
branches:
- 'main*'
- 'dev*'
workflow_dispatch:

jobs:


pytest:
runs-on: ubuntu-latest
if: |
github.event_name == 'push' ||
github.event.pull_request.draft == false
strategy:
fail-fast: false
matrix:
tokamak:
- C-MOD
- DIII-D
steps:

- name: Checkout
uses: actions/checkout@v4

- name: Add MDSplus sources
env:
GPG: http://www.mdsplus.org/dist/mdsplus.gpg.key
KEY: /usr/share/keyrings/mdsplus.asc
REPO: http://www.mdsplus.org/dist/Ubuntu22/repo
APT: /etc/apt/sources.list.d/mdsplus.list
run: |
curl -fsSL "$GPG" \
| sudo tee "$KEY"
echo "deb [arch=amd64 signed-by=$KEY] $REPO MDSplus alpha" \
| sudo tee "$APT"

- name: Update sources
run: sudo apt-get update

- name: Install prerequisites
run: |
sudo apt-get install msodbcsql17 # pre-installed
sudo apt-get install tdsodbc unixodbc # freetds
sudo apt-get install mdsplus-alpha-python # mdsplus

- name: Configure MDSplus
run: |
{
echo PYTHONPATH=/usr/local/mdsplus/python
echo LD_LIBRARY_PATH=/usr/local/mdsplus/lib
} \
| tee -a "$GITHUB_ENV"

- name: Setup SSH key
run: |
mkdir ~/.ssh
echo "${{ secrets.SSH_TUNNEL }}" \
| tee ~/.ssh/id_rsa \
| sha256sum
chmod 600 ~/.ssh/id_rsa

- name: Setup C-MOD
if: ${{ matrix.tokamak == 'C-MOD' }}
run: |
ssh "${{ secrets.CMOD_USER }}@${{ secrets.CMOD_HOST }}" \
-p "${{ secrets.CMOD_PORT }}" \
-L 8000:alcdata-archives:8000 \
-L 1433:alcdb2:1433 \
-o StrictHostKeyChecking=no \
-N -v 2> ssh.log &
echo "$!" \
| tee ssh.pid
echo "127.0.0.1 alcdata-archives alcdb2" \
| sudo tee -a /etc/hosts
echo "${{ secrets.CMOD_LOGIN }}" \
| tee ~/logbook.sybase_login \
| sha256sum
echo DISPY_TOKAMAK=CMOD \
| tee -a "$GITHUB_ENV"

- name: Setup DIII-D
if: ${{ matrix.tokamak == 'DIII-D' }}
env:
TDS: /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
run: |
ssh "${{ secrets.DIIID_USER }}@${{ secrets.DIIID_HOST }}" \
-p "${{ secrets.DIIID_PORT }}" \
-L 8000:atlas:8000 \
-L 8001:d3drdb:8001 \
-o StrictHostKeyChecking=no \
-N -v 2> ssh.log &
echo "$!" \
| tee ssh.pid
echo "127.0.0.1 atlas d3drdb" \
| sudo tee -a /etc/hosts
echo "${{ secrets.DIIID_LOGIN }}" \
| tee ~/D3DRDB.sybase_login \
| sha256sum
echo -e "[FreeTDS]\nDescription = FreeTDS\nDriver = $TDS" \
| sudo tee -a /etc/odbcinst.ini
echo DISPY_TOKAMAK=D3D \
| tee -a "$GITHUB_ENV"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.10'

- name: Install requirements
run: |
pip install -r .pip/main.txt -r .pip/dev.txt
pip install -e .

- name: Check tunnel
run: |
xargs -a ssh.pid ps -p
if ! grep 'Authenticated.*publickey' ssh.log
then
curl -s ipinfo.io && echo
cat -n ssh.log
false
fi

- name: Test SQL
run: python -O examples/sql.py

- name: Test MDSplus
run: python examples/mdsplus.py

- name: Test EFIT
run: python examples/efit.py

- name: Run all tests
run: pytest -v --durations=0 tests

- name: Close tunnel
run: xargs -a ssh.pid kill -TERM
Loading