refactor: pydantic lite rewrite #154

xgui3783 · 2022-01-25T13:46:46Z

No description provided.

codecov-commenter · 2022-01-25T14:57:22Z

Codecov Report

Merging #154 (fdc6e56) into main (c9085af) will increase coverage by 13.09%.
The diff coverage is 89.50%.

@@             Coverage Diff             @@
##             main     #154       +/-   ##
===========================================
+ Coverage   43.46%   56.55%   +13.09%     
===========================================
  Files          31       32        +1     
  Lines        3978     4238      +260     
===========================================
+ Hits         1729     2397      +668     
+ Misses       2249     1841      -408

Impacted Files	Coverage Δ
siibra/core/region.py	`65.67% <56.06%> (+15.48%)`	⬆️
siibra/commons.py	`66.66% <93.75%> (+5.76%)`	⬆️
siibra/core/space.py	`39.23% <94.11%> (+11.44%)`	⬆️
siibra/core/serializable_concept.py	`96.96% <96.96%> (ø)`
siibra/core/__init__.py	`100.00% <100.00%> (ø)`
siibra/core/atlas.py	`69.90% <100.00%> (+3.59%)`	⬆️
siibra/core/concept.py	`85.08% <100.00%> (+2.05%)`	⬆️
siibra/core/datasets.py	`86.73% <100.00%> (+8.95%)`	⬆️
siibra/core/parcellation.py	`74.76% <100.00%> (+7.74%)`	⬆️
siibra/features/connectivity.py	`54.63% <100.00%> (+24.92%)`	⬆️
... and 21 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9085af...fdc6e56. Read the comment docs.

remove unused imports

feat: added ieeg session to jsonable

feat: serialize connectivity feature

chore: unify receptor test

chore: add typing to features.query

without called super().__init_subclass__(), other super class's __init_subclass would not be called. In principle, *arg, **kwarg should also be added to __init_subclass__ method

chore: add id to connectivity matrix model chore: update NpArray to allow for kwarg init

xgui3783 · 2022-02-02T08:52:29Z

Dear @marcenko @skoehnen , I have finished the first step of pydantic lite rewrite of siibra-python.

All of the rewrite are additive (they are backwards compatible, and all new features are additive).

I would like to merge this into main ASAP, if you could spare a few minutes reviewing this PR?

skoehnen · 2022-02-02T13:06:27Z

@xgui3783 Changes in 176 files is a bit much, the GitHub diff viewer can't show them all without crashing (at least in Safari, need to check this in another browser).
Is there anything specific I should look out for?
Is there maybe a test I could run locally and check if the output jsons are openMINDS conform?

xgui3783 · 2022-02-02T13:36:08Z

@xgui3783 Changes in 176 files is a bit much, the GitHub diff viewer can't show them all without crashing (at least in Safari, need to check this in another browser)

Most are statically generated pydantic models (in siibra/openminds)

Is there anything specific I should look out for? Is there maybe a test I could run locally and check if the output jsons are openMINDS conform?

I have implemented some tests in test/core/* Unfortunately, mapping some entities to openminds 1-1 is non trivial. So we opted for using an intermediary model.

for example:

import siibra
jba29 = siibra.parcellations['2.9']
jba29_model = jba29.to_model()
list_jba29_vers = [jba29_ver.dict() for jba29_ver in jba29_model.brain_atlas_versions]

the list_jba29_vers should now be an array of dictionaries that conforms to brainAtlasVersion

I will see if I can hide changes to specific directory

xgui3783 · 2022-02-02T13:44:17Z

@skoehnen I added a .gitattribute file, which should hide all files & directories under siibra/openminds by default.

can you let me know if the PR still crashes the browser?

skoehnen · 2022-02-02T13:57:35Z

Weird, how come the tests suddenly fail.

skoehnen · 2022-02-02T14:05:04Z

Looks like a problem in test_receptor_to_model in test_receptors.py

skoehnen

FAILED test/features/test_receptors.py::test_receptor_to_model[receptor57]

Needs to be fixed.

siibra/features/connectivity.py

fix: np.int and np.float deprecation

xgui3783 · 2022-02-02T14:34:31Z

FAILED test/features/test_receptors.py::test_receptor_to_model[receptor57]

Needs to be fixed.

seems to be sporadically caused by unable to fetch KG metadata. reran the test resolves the issue.

skoehnen · 2022-02-02T14:36:17Z

FAILED test/features/test_receptors.py::test_receptor_to_model[receptor57]
Needs to be fixed.

seems to be sporadically caused by unable to fetch KG metadata. reran the test resolves the issue.

Okay, that is good, at least in the sense that it is not an error on our side ;)
I was a bit worried because there were some changes in that file.

skoehnen · 2022-02-02T14:41:20Z

@skoehnen I added a .gitattribute file, which should hide all files & directories under siibra/openminds by default.

can you let me know if the PR still crashes the browser?

No, it didn't change anything here. But I guess new PRs will not show the generated files.

I fixed it for me by using "Jump To" and looking at specific files and then reloading ;)

skoehnen · 2022-02-02T14:45:31Z

Green across the board, looks good to me.

I will take a deeper look into the generated models and their "openMINDS conformity" when I am finished with the curation migration.

marcenko

Seems fine to me

xgui3783 added 8 commits January 17, 2022 17:55

experimental: pydantic lite

4d10af5

improve typing of registry

5f83b8b

tests: space -> json

8ce4602

pydantic lite rewrite of region

34f6c4b

Merge remote-tracking branch 'origin/main' into feat_pydanticAddition

a42537c

init __init__.py's in openminds

ac18843

pydantice lite rewrite of atlas & parcellation

ee80c8d

update setup.py and requirements

03bc032

xgui3783 added 2 commits January 25, 2022 15:59

ignore F722 lint rule

9b35859

codecov ignore all in siibra/openminds

3f9d5a3

xgui3783 mentioned this pull request Jan 25, 2022

Refactor/pydantic #143

Closed

xgui3783 added 13 commits January 25, 2022 18:21

minor tweak to global base models

8c23218

minor fix to model

c19f715

remove unused imports

minor fix to model aliases

19df421

bugfix: context alias

c8a0534

chore: restructure jsonable concept

94806ad

feat: added ieeg session to jsonable

chore: move np array serialization to concept

d96be7c

feat: serialize connectivity feature

chore: update parcellation test to only xfail when fail

311239f

chore: unify receptor test

feat: add volume & voi to serializable

60f1c7b

chore: white spaces

9713fdd

bugfix: add id to dataset model

9857e75

chore: add typing to features.query

bugfix: __init_subclass__ fix

ce46231

without called super().__init_subclass__(), other super class's __init_subclass would not be called. In principle, *arg, **kwarg should also be added to __init_subclass__ method

bugfix: typing for ieeg session to_model method

167f2fc

chore: add id to ieeg session model

7457bab

chore: add id to connectivity matrix model chore: update NpArray to allow for kwarg init

xgui3783 marked this pull request as ready for review February 1, 2022 09:04

feat: add detail flag to receptor feature to_model

d3720b7

xgui3783 requested review from marcenko and skoehnen February 2, 2022 08:49

add gitattributes to all openminds models

000a28d

skoehnen suggested changes Feb 2, 2022

View reviewed changes

siibra/features/connectivity.py Outdated Show resolved Hide resolved

siibra/features/connectivity.py Outdated Show resolved Hide resolved

fix: region.parent check

1589b33

fix: np.int and np.float deprecation

skoehnen approved these changes Feb 2, 2022

View reviewed changes

marcenko approved these changes Feb 2, 2022

View reviewed changes

xgui3783 added 7 commits February 4, 2022 10:05

feat: add parcellation specific pev/pe model

1ae945c

fix up has_versions of to_pe_model method

ddb72b5

improve id extraction of pe has_version

412c43b

fix PE name

e12e5db

update superficial bundle PE generation

ede23d8

update swma PE/PEV serialization

faf200b

Merge branch 'main' into feat_pydanticAddition

fdc6e56

xgui3783 mentioned this pull request Feb 9, 2022

feat: adopt pydantic-lite rewrite FZJ-INM1-BDA/siibra-api#67

Merged

xgui3783 merged commit ba51d9f into main Feb 10, 2022

xgui3783 deleted the feat_pydanticAddition branch February 10, 2022 14:38

xgui3783 mentioned this pull request Feb 11, 2022

Add jsonable abstract class or decorators for easy json encoding #101

Closed

xgui3783 mentioned this pull request Mar 2, 2022

Add explicit openMINDS support for basic concepts #134

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: pydantic lite rewrite #154

refactor: pydantic lite rewrite #154

xgui3783 commented Jan 25, 2022

codecov-commenter commented Jan 25, 2022 •

edited

Loading

xgui3783 commented Feb 2, 2022

skoehnen commented Feb 2, 2022 •

edited

Loading

xgui3783 commented Feb 2, 2022

xgui3783 commented Feb 2, 2022

skoehnen commented Feb 2, 2022

skoehnen commented Feb 2, 2022

skoehnen left a comment

xgui3783 commented Feb 2, 2022

skoehnen commented Feb 2, 2022

skoehnen commented Feb 2, 2022

skoehnen commented Feb 2, 2022 •

edited

Loading

marcenko left a comment

refactor: pydantic lite rewrite #154

refactor: pydantic lite rewrite #154

Conversation

xgui3783 commented Jan 25, 2022

codecov-commenter commented Jan 25, 2022 • edited Loading

Codecov Report

xgui3783 commented Feb 2, 2022

skoehnen commented Feb 2, 2022 • edited Loading

xgui3783 commented Feb 2, 2022

xgui3783 commented Feb 2, 2022

skoehnen commented Feb 2, 2022

skoehnen commented Feb 2, 2022

skoehnen left a comment

Choose a reason for hiding this comment

xgui3783 commented Feb 2, 2022

skoehnen commented Feb 2, 2022

skoehnen commented Feb 2, 2022

skoehnen commented Feb 2, 2022 • edited Loading

marcenko left a comment

Choose a reason for hiding this comment

codecov-commenter commented Jan 25, 2022 •

edited

Loading

skoehnen commented Feb 2, 2022 •

edited

Loading

skoehnen commented Feb 2, 2022 •

edited

Loading