[MRG, IO] Persyst reader into Raw object #8176

adam2392 · 2020-08-31T17:36:34Z

Reference issue

Closes: #8173

What does this implement/fix?

Adds a Persyst reader for .lay + .dat file combinations and unit tests.

Note:

birthdate and measurement date is not a strict adherence sometimes when output by Persyst. I am not sure why... but things like 16-Nov-199 obviously probably stands for nov 16th, 1992 based on the data, but not sure why it gets output funky. I'm assuming these things can change with Persyst versions without our knowing.

With that being said though, the data format and such seems like it shouldn't change all of a sudden.

Additional information

Updates the testing repo to: mne-tools/mne-testing-data#73

adam2392 · 2020-09-02T02:13:07Z

This is ready for review/merge on my end.

I am not sure why that final CI test failed. It says it has to do with unpacking the tar file(?)

mne/io/persyst/persyst.py

mne/io/persyst/tests/test_persyst.py

larsoner

Looks like it's close to done, despite a number of comments/ideas I've written!

doc/_includes/data_formats.rst

doc/changes/latest.inc

mne/io/persyst/persyst.py

mne/io/persyst/tests/test_persyst.py

agramfort

looks good !

@adam2392 you should also see then how mne-bids allows you to write such files in bids

adam2392 · 2020-09-03T12:45:56Z

This should be good to go assuming the CIs turn green and @larsoner is okay with my changes to address his comments :). Thanks!

adam2392 · 2020-09-03T20:26:49Z

Just wanted to confirm that this is "good to go", possibly waiting for some additional CI test from @larsoner in #8210?

I am a bit noob in understanding what _mult_cal_one does, but I presume it works since it passed the _test_raw tests :p.

larsoner · 2020-09-04T00:13:41Z

Yes let's wait for #8210. I just got it working (I think!). Do you want me to push a merge commit once #8210 is in master so we can see if everything is good here?

mne/io/persyst/persyst.py

larsoner · 2020-09-04T12:40:09Z

Looks like the channel with index 55 (name 'pol $tp9') has values all right around [-0.01200273], was this a bad channel during the recording or something?

adam2392 · 2020-09-04T14:14:07Z

Looks like the channel with index 55 (name 'pol $tp9') has values all right around [-0.01200273], was this a bad channel during the recording or something?

That is a byproduct of some center's recording systems I work with. So a rule of thumb I've come to realize is that if there is a $ sign, then it most likely means its a dead channel (not connected, or used as REF, etc.)

Co-authored-by: Eric Larson <[email protected]>

agramfort

@larsoner merge if you're happy

larsoner · 2020-09-06T02:22:43Z

Thanks @adam2392 !

* upstream/master: (489 commits) MRG, DOC: Fix ICA docstring, add whitening (mne-tools#8227) MRG: Extract measurement date and age for NIRX files (mne-tools#7891) Nihon Kohden EEG file reader WIP (mne-tools#6017) BUG: Fix scaling for src_mri_t in coreg (mne-tools#8223) MRG: Set pyvista as default 3d backend (mne-tools#8220) MRG: Recreate our helmet graphic (mne-tools#8116) [MRG] Adding get_montage for montage to BaseRaw objects (mne-tools#7667) ENH: Allow setting tqdm backend (mne-tools#8177) [MRG, IO] Persyst reader into Raw object (mne-tools#8176) MRG, BUG: Fix errors in IO/loading/projectors (mne-tools#8210) MAINT: vectorize _read_annotations_edf (mne-tools#8214) FIX : events_from_annotation when annotations.orig_time is None and f… (mne-tools#8209) FIX: do not project to sphere; DOC - explain how to get EEGLAB-like topoplots (mne-tools#7455) [MRG, DOC] Added linear algebra of transform to doc (mne-tools#7087) FIX: Travis failure on python3.8.1 (mne-tools#8207) BF: String formatting in exception message (mne-tools#8206) BUG: Fix STC limit bug (mne-tools#8202) MRG, DOC: fix ica tutorial (mne-tools#8175) CSP component order selection (mne-tools#8151) MRG, ENH: Add on_missing to plot_events (mne-tools#8198) ...

* Initial commit. * Initial persyst commit. * Making reader work. * Adding example. * removing extra diff. * removing white space? * removing white space? * Trying to fix flake8. * Fix pydocstyle. * fixing typo. * Adding updated section for persyst. * Fix pep. * Fixing example docstring. * Removing example. * Unit testing passed. * Fixing unit test. * Adding updated tests. * Making changes suggested by larsoner. * Update mne/io/persyst/tests/test_persyst.py Co-authored-by: Eric Larson <[email protected]> * Update mne/io/persyst/tests/test_persyst.py Co-authored-by: Eric Larson <[email protected]> * Fixing cal. * Adding updated persyst. * Remove unused line. * Update mne/io/persyst/persyst.py Co-authored-by: Eric Larson <[email protected]> * Fixing dictionary mapping. * Adding updated doc. * Fixing pep. * Fixing subject map. * Keeping channels upper case. * Fixing coverage. * Fix docstring. Co-authored-by: Eric Larson <[email protected]>

adam2392 changed the title ~~Persyst~~ [WIP] Persyst reader into Raw object Aug 31, 2020

adam2392 changed the title ~~[WIP] Persyst reader into Raw object~~ [MRG, IO] Persyst reader into Raw object Sep 2, 2020

adam2392 force-pushed the persyst branch from 6e78415 to 6f500f2 Compare September 2, 2020 02:22

agramfort reviewed Sep 2, 2020

View reviewed changes

mne/io/persyst/persyst.py Outdated Show resolved Hide resolved

mne/io/persyst/persyst.py Outdated Show resolved Hide resolved

mne/io/persyst/tests/test_persyst.py Show resolved Hide resolved

larsoner reviewed Sep 2, 2020

View reviewed changes

agramfort approved these changes Sep 2, 2020

View reviewed changes

adam2392 mentioned this pull request Sep 2, 2020

[API] Writing other data formats directly into BIDS (e.g. Persyst) mne-tools/mne-bids#544

Closed

larsoner mentioned this pull request Sep 3, 2020

MRG, BUG: Fix errors in IO/loading/projectors #8210

Merged

larsoner added this to the 0.21 milestone Sep 4, 2020

larsoner reviewed Sep 4, 2020

View reviewed changes

mne/io/persyst/persyst.py Outdated Show resolved Hide resolved

mne/io/persyst/persyst.py Outdated Show resolved Hide resolved

mne/io/persyst/persyst.py Show resolved Hide resolved

adam2392 force-pushed the persyst branch from 4ee361c to b62c852 Compare September 4, 2020 14:20

adam2392 added 13 commits September 4, 2020 11:01

Initial commit.

ca71037

Initial persyst commit.

33c5600

Making reader work.

bb42cb9

Adding example.

ffe0acf

removing extra diff.

f8fbc86

removing white space?

0f541ac

removing white space?

cad1cc5

Trying to fix flake8.

776bb2f

Fix pydocstyle.

8358da4

fixing typo.

8ecb5ad

Adding updated section for persyst.

4ce5c7a

Fix pep.

4969505

Fixing example docstring.

6622a8b

adam2392 and others added 13 commits September 4, 2020 11:01

Removing example.

15b3b3c

Unit testing passed.

b6c435b

Fixing unit test.

2268649

Adding updated tests.

2c4f4df

Making changes suggested by larsoner.

e77656e

Update mne/io/persyst/tests/test_persyst.py

751ad21

Co-authored-by: Eric Larson <[email protected]>

Update mne/io/persyst/tests/test_persyst.py

c9e9033

Co-authored-by: Eric Larson <[email protected]>

Fixing cal.

129dc6b

Adding updated persyst.

d96a5c5

Remove unused line.

9208eb5

Update mne/io/persyst/persyst.py

7db827f

Co-authored-by: Eric Larson <[email protected]>

Fixing dictionary mapping.

f51039e

Adding updated doc.

65000df

adam2392 force-pushed the persyst branch from 5f7f97f to 65000df Compare September 4, 2020 15:02

adam2392 mentioned this pull request Sep 4, 2020

[MRG] Add Persyst Reader To Convert to BV In BIDS mne-tools/mne-bids#546

Merged

6 tasks

adam2392 added 5 commits September 4, 2020 11:42

Fixing pep.

3a7cbe2

Fixing subject map.

171792e

Keeping channels upper case.

891c677

Fixing coverage.

16ccfe6

Fix docstring.

b53f167

agramfort approved these changes Sep 5, 2020

View reviewed changes

larsoner merged commit 1cda89a into mne-tools:master Sep 6, 2020

hoechenberger mentioned this pull request Sep 16, 2020

MAINT: Release 0.21 #8150

Closed

larsoner mentioned this pull request Oct 10, 2020

ENH: read mff evoked #8354

Merged

adam2392 deleted the persyst branch October 30, 2020 16:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MRG, IO] Persyst reader into Raw object #8176

[MRG, IO] Persyst reader into Raw object #8176

adam2392 commented Aug 31, 2020 •

edited

Loading

adam2392 commented Sep 2, 2020

larsoner left a comment

agramfort left a comment

adam2392 commented Sep 3, 2020

adam2392 commented Sep 3, 2020 •

edited

Loading

larsoner commented Sep 4, 2020

larsoner commented Sep 4, 2020

adam2392 commented Sep 4, 2020

agramfort left a comment

larsoner commented Sep 6, 2020

[MRG, IO] Persyst reader into Raw object #8176

[MRG, IO] Persyst reader into Raw object #8176

Conversation

adam2392 commented Aug 31, 2020 • edited Loading

Reference issue

What does this implement/fix?

Additional information

adam2392 commented Sep 2, 2020

larsoner left a comment

Choose a reason for hiding this comment

agramfort left a comment

Choose a reason for hiding this comment

adam2392 commented Sep 3, 2020

adam2392 commented Sep 3, 2020 • edited Loading

larsoner commented Sep 4, 2020

larsoner commented Sep 4, 2020

adam2392 commented Sep 4, 2020

agramfort left a comment

Choose a reason for hiding this comment

larsoner commented Sep 6, 2020

adam2392 commented Aug 31, 2020 •

edited

Loading

adam2392 commented Sep 3, 2020 •

edited

Loading