Release/2.8.0 #198

YaphetKG · 2022-02-03T18:14:01Z

Creating release branch to merge into master, consisting latest develop changes

* support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]>

updating version for final push to master

stevencox · 2022-02-04T19:27:02Z

@YaphetKG - could we please get an additional comment above this code? I'm not sure what it's doing or why it's needed.

Adding more comments

YaphetKG · 2022-02-07T14:34:18Z

@YaphetKG - could we please get an additional comment above this code? I'm not sure what it's doing or why it's needed.

@stevencox added more comments. I wanted to have this config re-parsing to be able to add some other types of filter in the future for cases that we might have "biolink:Publication" nodes that might come from different sources eg. CDE (current), Phenx, or others. We might want to cast them to their respective types. Since config is just a dict currently, and in roger we store it as a yaml, this function would provide a layer to convert those as lambda functions or other way to apply those conditional filters.

But currently it just does reformatting of the original config.

waTeim · 2022-02-07T21:25:00Z

The extremely minor question: Are we concerned at all about 2 different things differing only in non-alphanumerics in to_snakecase being converted to the same thing since all of those ---> _

YaphetKG · 2022-02-08T13:58:24Z

The extremely minor question: Are we concerned at all about 2 different things differing only in non-alphanumerics in to_snakecase being converted to the same thing since all of those ---> _

in biolink entity classes have curies that are of the format biolink:PascalCase (biolink ref.) . In tranql's query language we use snake case with out the biolink: prefix. But when tranql responds back it uses the biolink:PascalCase format in the response graph. This method allows conversion between these two formats.

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]>

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]>

* Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]>

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]>

* Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]>

* Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]>

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]>

* Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]>

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * version bump Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]>

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * sprint parser * adding parser to plugin manager * sprint form names parse from file name * moving sprint to sprint parser * remove old import * empty desc Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]>

* Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Master to develop sync (#262) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * version bump Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * Sprint (#264) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * sprint parser * adding parser to plugin manager * sprint form names parse from file name * moving sprint to sprint parser * remove old import * empty desc Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]>

* Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Master to develop sync (#262) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * version bump Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * Sprint (#264) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parse…

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * upgrade libs * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]>

* Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Master to develop sync (#262) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * version bump Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * Sprint (#264) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parse…

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * Release/2.9.5 (#266) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Master to develop sync (#262) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. …

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * async search api and elastic search interface * bump version * add root path * fix result * fix query models * fix query models * return concept scores * remove un-used import * remove cruft api and schema * shutdown event disconnect elastic * fix topmed kgx structure * Bump mistune from 0.8.4 to 2.0.3 (#253) Bumps [mistune](https://github.com/lepture/mistune) from 0.8.4 to 2.0.3. - [Release notes](https://github.com/lepture/mistune/releases) - [Changelog](https://github.com/lepture/mistune/blob/master/docs/changes.rst) - [Commits](lepture/mistune@v0.8.4...v2.0.3) --- updated-dependencies: - dependency-name: mistune dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix exclude --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Master to develop sync (#262) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * version bump Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * Sprint (#264) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_pars…

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * validation search functions * modify factory * code clean up remove nboost * remove search sync * fix tests, seperate indexing from search object * fix dependabot alerts --------- Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]>

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * Release/2.9.5 (#266) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Master to develop sync (#262) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree vari…

* Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Master to develop sync (#262) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * version bump Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * Sprint (#264) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_pars…

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * Release/2.9.5 (#266) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Master to develop sync (#262) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * P…

* Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * update version * remove cruft from merge Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Hoid <[email protected]> * Release/2.9.5 (#266) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Bump version. * Revert version to dug format. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> * Add github creds env var. * Fix version typo. * Initial commit * Reduce ephemeral storage limits and requests * More parsers (#248) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * consolidate dbgap format parser in single file , adds crdc and kfdrc parsers * adding tests * bump version * parser when versions of studies are > 9 * test for version * fix long text issues, and encoding errors * nltk initialization * change nltk approach for sliding window Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Master to develop sync (#262) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Release/2.9.1 (#205) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Release/2.9.1 Renames SPARC datasets as SPARC instead of dbgap Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> * Update _version.py (#206) * Release/2.9.2 (#209) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release 2.9.3 (#244) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. * Programatically remove quotes from version tag. * dug dump concepts api created and tested (#229) Co-authored-by: Nathan Braswell <[email protected]> * Update _version.py (#234) * Version changes + separate build and publish. * Semantic versioning prep. * Add develop and master versioning and tagging. * Ncpi index fix (#232) * Renamed anvil to ncpi * Update ncpi datasets catalog * Modified script to download NCPI datasets into platform subfolders * Updated NCPI integration dataset * Removed unused variable * Removed ncpi top level folder to spread results among subfolders * Change output dir to data instead of ncpi subdir * Moved NCPI subdirs into main data folder for ingest as per Yaphet's request Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Howard Lander <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> Co-authored-by: Charles Bennett <[email protected]> Co-authored-by: Nathaniel Braswell <[email protected]> Co-authored-by: Nathan Braswell <[email protected]> Co-authored-by: cnbennett3 <[email protected]> Co-authored-by: Alex Waldrop <[email protected]> * Release/2.9.4 (#260) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Dev version bump (#202) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Attribute mapping from node to dug element (#203) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * adding more config options for node extraction * some refactoring Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Changed DbGaP to SPARC in the scicrunch parser (#204) * Anvil (#207) * Added updated anvil dataset catalog * Added script for downloading all anvil data dicts * Added current anvil data dictionaries to data folder to be used for indexing * Anvil parser (#208) * Release/2.8.0 (#198) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Update _version.py * Update _version.py updating version for final push to master * Update factory.py Adding more comments Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> * Release/v2.9.0 (#201) * Bumping version * support for extracting dug elements from graph (#197) * support for extracting dug elements from graph * adding flag for enabling dug element extraction from graph * adding new config for node_to dug element parsing * adding more parameters to crawler to able configuration to element extraction logic * add tests * add tests for crawler Co-authored-by: Yaphetkg <[email protected]> * Display es scores (#199) * Include ES scores in variable results * Round ES score to 6 * Update _version.py (#200) * Update _version.py Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * anvil parser * bump number of files test * Update dbgap_parser.py * Update anvil_dbgap_parser.py change to AnVIL * Update test_parsers.py update test Co-authored-by: Carl Schreep <[email protected]> Co-authored-by: Yaphetkg <[email protected]> Co-authored-by: Ginnie Hench <[email protected]> * Initial Kaniko build. * Move version file definition. * Quote env vars. * Update env vars. * Update env vars. * Update env vars. * env var changes. * env var changes. * env var changes. * env var changes. * Update DOCKER_IMAGE var. * Update DOCKER_IMAGE var in kaniko cmd. * Update kaniko destination line. * Update kaniko destination line. * Moree variable madness. …

Carl Schreep and others added 3 commits December 2, 2021 15:02

Bumping version

40ac323

Update _version.py

79ac646

YaphetKG requested review from alexwaldrop and waTeim February 3, 2022 18:14

alexwaldrop approved these changes Feb 3, 2022

View reviewed changes

Update _version.py

cf19e46

updating version for final push to master

YaphetKG requested a review from stevencox February 4, 2022 18:00

stevencox approved these changes Feb 4, 2022

View reviewed changes

Update factory.py

532739a

Adding more comments

waTeim approved these changes Feb 8, 2022

View reviewed changes

YaphetKG merged commit 60b473a into master Feb 8, 2022

YaphetKG mentioned this pull request Jul 19, 2023

Master back to dev (#297) #298

Merged

YaphetKG deleted the release/2.8.0 branch January 25, 2024 18:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release/2.8.0 #198

Release/2.8.0 #198

YaphetKG commented Feb 3, 2022

stevencox commented Feb 4, 2022

YaphetKG commented Feb 7, 2022 •

edited

Loading

waTeim commented Feb 7, 2022

YaphetKG commented Feb 8, 2022 •

edited

Loading

Release/2.8.0 #198

Release/2.8.0 #198

Conversation

YaphetKG commented Feb 3, 2022

stevencox commented Feb 4, 2022

YaphetKG commented Feb 7, 2022 • edited Loading

waTeim commented Feb 7, 2022

YaphetKG commented Feb 8, 2022 • edited Loading

YaphetKG commented Feb 7, 2022 •

edited

Loading

YaphetKG commented Feb 8, 2022 •

edited

Loading