[WIP] rearranging BIDSMRI2NIDM command

[djarecka]

• creating a new class BidsMriProject
• moving most of the script from BIDSMRI2NIDM.py to the new class;
• removing bidsignore arg and changing outpufie to outputdir

[djarecka]

@dbkeator - please do not merge, this is work in progress, just wanted to push it before our today meeting

[djarecka]

@dbkeator - I did some more rearrangement, adding functions etc.

I also added click for commands (it can be used now pybids bidsmri2nidm) and tried to implement logic we discussed yesterday. Unfortunately couldn't find the way to ask click to check automatically if one option requires others, so doing this manually at the beginning.
I
'm not sure if everything works as it should, e.g. I don't see a new json file for mapping. But when I run your original script like that:
python BIDSMRI2NIDM_orig.py -d ~/datasets/abide/RawDataBIDS/CMU_a --json_map tests_json_map --github djarecka ddd
I also can't see any new file, so I'm missing something important.

Happy to chat today/tomorrow to clarify.

[dbkeator]

@djarecka I can chat tomorrow after 10am PST. Are you available tomorrow ?

[dbkeator]

@djarecka Here's what should happen:

If one runs the program and supplies a JSON variable->term mapping file then load it and try and mach the variables to terms. If if can't find a variable it needs in the JSON mapping file then it goes through the query Interlex, NIDM-Exp owl file, and github local term definition loop. At each iteration when one selects a new variable-> term mapping it adds it to the JSON mapping file and re-saves it (in case the program stops or one exits prematurely they won't have to go back and the the mappings they've already completed again). If one doesn't supply a JSON mapping file but provides the -ilxkey and -github parameters (remember these need to be supplied together) then the program will save the JSON mapping file as "participants.json" in the root BIDS directory specified by the -d parameter. Note, we'll want to change this a bit once I add code to handle phenotype directories in BIDS datasets. In that case we'll want to save the corresponding variable->term mappings in JSON files for each corresponding phenotype.tsv file....

[dbkeator]

@djarecka ...Also I think the command should be pynidm bidsmri2nidm (not pybids ....maybe that was a typo)

[djarecka]

Yes, that was a typo. I'm on my way to the airport. Will read your comments later, but won't be able to talk tomorrow. We can talk on Monday during or after the call.

…

On Thu, Aug 16, 2018, 15:17 David Keator ***@***.***> wrote: @djarecka <https://github.com/djarecka> ...Also I think the command should be pynidm bidsmri2nidm (not pybids ....maybe that was a typo) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#99 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABlfGkKNHtx0LvFsqSwTJ2mNiThtS8d7ks5uRcU-gaJpZM4V6uiC> .

[satra]

just a thought - can we make the command be called nidmcli or just nidm?

[djarecka]

@satra - sure

…

On Thu, Aug 16, 2018, 18:35 Satrajit Ghosh ***@***.***> wrote: just a thought - can we make the command be called nidmcli or just nidm? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#99 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABlfGocX7Iv1JGMd_l6mu0peFbWs7OE_ks5uRfOfgaJpZM4V6uiC> .

[djarecka]

@dbkeator - what should happen if the json mapping file is not provided and the participants.json is also not present? This is the case for abide/RawDataBIDS/CMU_a
(that also might answer question whether participants.json is required by BIDS, it's not present in this dataset)

[dbkeator]

@djarecka If the user doesn't provide a JSON mapping file and they add the -ilxkey -github -owl parameters. I think we want to make those jointly required so if one wants to do variable->term mapping then they would provide -ilxkey [api key] + -github [username token] and the -owl parameter is pointless and should just be the default if -ilxkey and -github are provided because what we're doing is then searching the Interlex and the NIDM OWL files for fuzzy matches for the variable names. The -github parameter is used when one doesn't find an appropriate term and thus defines their own in the github repo. Note, we'll be changing that so one can push new terms directly to Interlex but that's in the near future.

So, to continue, if one provides the -ilxkey + -github stuff then the code will do variable -> term mappings and save a new JSON mapping file. Right now it's defaulting to "participants.json" (see

PyNIDM/nidm/experiment/tools/BIDSMRI2NIDM.py

Line 239 in bb7d70c

if args.json_map == None:

)

[djarecka]

@dbkeator - you can try using:
pynidm bidsmri2nidm -d ~/datasets/adhd200/RawDataBIDS/Brown --png or
pynidm bidsmri2nidm -d ~/datasets/adhd200/RawDataBIDS/Brown --png --mapping --ilxkey [your key] or

pynidm bidsmri2nidm -d ~/datasets/adhd200/RawDataBIDS/Brown --png --mapping --ilxkey [your key] --github_username [your username] or
pynidm bidsmri2nidm -d ~/datasets/adhd200/RawDataBIDS/Brown --png --mapping --ilxkey [your key] --github_username [your username] --github_token [the token]

[dbkeator]

@djarecka: I think you're missing some libraries in bids_base.py:

from prov.model import QualifiedName
from prov.model import Namespace as provNamespace
from nidm.experiment.Core import Core

Traceback (most recent call last):
File "/Users/dbkeator/anaconda/envs/python35/bin/pynidm", line 11, in
load_entry_point('nidm==0.1.dev0', 'console_scripts', 'pynidm')()
File "/Users/dbkeator/anaconda/envs/python35/lib/python3.5/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/Users/dbkeator/anaconda/envs/python35/lib/python3.5/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/Users/dbkeator/anaconda/envs/python35/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/dbkeator/anaconda/envs/python35/lib/python3.5/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/dbkeator/anaconda/envs/python35/lib/python3.5/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/Users/dbkeator/anaconda/envs/python35/lib/python3.5/site-packages/nidm-0.1.dev0-py3.5.egg/nidm/experiment/tools/BIDSMRI2NIDM.py", line 132, in bidsmri2nidm
bidsmri = BidsMriNidm(directory, json_map, github, ilxkey, owl)
File "/Users/dbkeator/anaconda/envs/python35/lib/python3.5/site-packages/nidm-0.1.dev0-py3.5.egg/nidm/experiment/bidsmri.py", line 12, in init
github=github, key=key, owl=owl)
File "/Users/dbkeator/anaconda/envs/python35/lib/python3.5/site-packages/nidm-0.1.dev0-py3.5.egg/nidm/experiment/bids_base.py", line 22, in init
self._create_session_participants()
File "/Users/dbkeator/anaconda/envs/python35/lib/python3.5/site-packages/nidm-0.1.dev0-py3.5.egg/nidm/experiment/bids_base.py", line 144, in _create_session_participants
acq_entity.add_attributes({QualifiedName(provNamespace(Core.safe_string(None,string=str(key)), column_to_terms[key]["url"]), ""):value})
NameError: name 'QualifiedName' is not defined

[dbkeator]

@djarecka Besides the issue above https://github.com/incf-nidash/PyNIDM/pull/99#issuecomment-416763275 everything else works. Once that's corrected we can merge....

[djarecka]

@dbkeator - could you please send me the link again, it doesn't work for me

[djarecka]

@dbkeator - I'm still not sure how did you get the error you posted. did you run one of the command I suggested?