Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new command nf-core rocrate to create a Research Object (RO) crate for a pipeline #2680

Merged
merged 72 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
e7249b7
second attempt to add ro crates
mashehu Jan 16, 2024
a597c8c
fix import
mashehu Jan 16, 2024
7a66e06
first running version
mashehu Jan 23, 2024
1afb40c
Merge branch 'dev' of github.com:nf-core/tools into add-rocreate
mashehu Jan 24, 2024
ddaacbd
[automated] Update CHANGELOG.md
nf-core-bot Jan 24, 2024
ffcdc6f
fix types
mashehu Jan 24, 2024
a87f6aa
fix types
mashehu Jan 24, 2024
63cccc8
use git to get all workflow files
mashehu Jan 24, 2024
e65248e
restructure code and add tests
mashehu Jan 24, 2024
c11b536
Merge branch 'add-rocreate' of github.com:mashehu/tools into add-rocr…
mashehu Jan 24, 2024
1210d1e
add missing dep
mashehu Jan 24, 2024
83d2217
add documentation
mashehu Jan 25, 2024
b505a62
add recommendations from comments
mashehu Jan 25, 2024
e8959ef
add git contributors
mashehu Jan 25, 2024
c271c75
Add to files_exist linting
mashehu Jan 25, 2024
02b2c4d
fix tests
mashehu Jan 25, 2024
43fe299
fake ro-crate file for lint tests
mashehu Jan 25, 2024
b0ed038
fix filename for CI
mashehu Jan 25, 2024
24510be
better help hint message
mashehu Jan 25, 2024
3bd0eaf
fix class name
mashehu Jan 25, 2024
97d298a
add correct parent directory to faked crate
mashehu Jan 25, 2024
d6b7e6b
Add a fake RO crate file to pass lint tests, rewrite with pathlib
mashehu Jan 25, 2024
0551173
add empty ro-crate after sync
mashehu Jan 25, 2024
59767c3
fix tests
mashehu Jan 25, 2024
440168c
fix linting tests
mashehu Jan 25, 2024
d0e03b1
add linting test for new hint
mashehu Jan 25, 2024
61c898a
Merge branch 'dev' of github.com:nf-core/tools into add-rocreate
mashehu Jul 31, 2024
eae9a97
fix tests
mashehu Jul 31, 2024
76b6d67
fix ci test
mashehu Jul 31, 2024
12529a5
use github to guess author name, set names as ids if no orcid
mashehu Jul 31, 2024
ff9df16
add bioschemas, datecreated and datemodified (only set to current tim…
mashehu Jul 31, 2024
9f7dccd
follow `ComputationalWorkflow` schema for main entitty.
mashehu Aug 5, 2024
fd54944
fix date created field
mashehu Aug 5, 2024
b875e7e
add "about" field to workflow diagram
mashehu Aug 5, 2024
c304ab2
add input as separate entity
mashehu Aug 5, 2024
ba3bd83
add version to main entity
mashehu Aug 5, 2024
ec7a334
update snapshot
mashehu Aug 5, 2024
501aa74
update snapshot with py 3.12
mashehu Aug 5, 2024
26cf2e8
update snapshots in codespaces
mashehu Aug 5, 2024
2708194
Merge branch 'dev' of github.com:nf-core/tools into add-rocreate
mashehu Aug 16, 2024
e1061de
fix incorrect type
mashehu Aug 16, 2024
7793a9f
switch to add_workflow method to add main entity and add components a…
mashehu Aug 16, 2024
f54becb
ro_crate -> rocrate
mashehu Aug 27, 2024
5134ed1
Merge branch 'dev' into add-rocreate
mashehu Aug 27, 2024
6645e4c
second attempt to add ro crates
mashehu Jan 16, 2024
553135a
fix import
mashehu Jan 16, 2024
bf30f0d
first running version
mashehu Jan 23, 2024
9ed053a
restructure code and add tests
mashehu Jan 24, 2024
56a170c
add missing dep
mashehu Jan 24, 2024
8419e9f
add recommendations from comments
mashehu Jan 25, 2024
bdbc045
add git contributors
mashehu Jan 25, 2024
3ba54c4
fix filename for CI
mashehu Jan 25, 2024
9a66883
better help hint message
mashehu Jan 25, 2024
a656617
fix class name
mashehu Jan 25, 2024
fb0d091
add correct parent directory to faked crate
mashehu Jan 25, 2024
2469697
add empty ro-crate after sync
mashehu Jan 25, 2024
f521fa3
fix tests
mashehu Jul 31, 2024
797fefb
fix ci test
mashehu Jul 31, 2024
9d5251f
use github to guess author name, set names as ids if no orcid
mashehu Jul 31, 2024
a605ae6
add bioschemas, datecreated and datemodified (only set to current tim…
mashehu Jul 31, 2024
6c6b8c9
follow `ComputationalWorkflow` schema for main entitty.
mashehu Aug 5, 2024
c2f837c
fix date created field
mashehu Aug 5, 2024
0778b2a
add "about" field to workflow diagram
mashehu Aug 5, 2024
ccfd6d9
add input as separate entity
mashehu Aug 5, 2024
776c6cc
add version to main entity
mashehu Aug 5, 2024
a50f12f
fix incorrect type
mashehu Aug 16, 2024
49592cb
switch to add_workflow method to add main entity and add components a…
mashehu Aug 16, 2024
c1d08d7
ro_crate -> rocrate
mashehu Aug 27, 2024
5e4b4f3
use repo2crate to generate the main structure of the crate
mashehu Oct 17, 2024
4616629
Merge branch 'add-rocreate' of github.com:mashehu/tools into add-rocr…
mashehu Oct 17, 2024
2993fbb
Merge branch 'dev' of github.com:nf-core/tools into add-rocreate
mashehu Oct 23, 2024
ad7b895
remove outdated file
mashehu Oct 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
- Add new subcommand: `nf-core logo-create` to output an nf-core logo for a pipeline (instead of going through the website) ([#2662](https://github.com/nf-core/tools/pull/2662))
- Update actions/cache action to v4 ([#2666](https://github.com/nf-core/tools/pull/2666))
- Remove redundanct v in pipeline version for emails ([#2667](https://github.com/nf-core/tools/pull/2667))
- Add new command `nf-core rocrate` to create a Research Object (RO) crate for a pipeline ([#2680](https://github.com/nf-core/tools/pull/2680))

# [v2.11.1 - Magnesium Dragon Patch](https://github.com/nf-core/tools/releases/tag/2.11) - [2023-12-20]

Expand Down
36 changes: 34 additions & 2 deletions nf_core/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,10 @@
"modules",
"subworkflows",
"schema",
"create-logo",
"bump-version",
"sync",
"rocrate",
"create-logo",
],
},
],
Expand Down Expand Up @@ -1855,7 +1856,7 @@ def validate(pipeline, params):
@click.option(
"--url",
type=str,
default="https://nf-co.re/pipeline_schema_builder",
default="https://oldsite.nf-co.re/pipeline_schema_builder",
help="Customise the builder URL (for development work)",
)
def build(dir, no_prompts, web_only, url):
Expand Down Expand Up @@ -2124,6 +2125,37 @@ def sync(dir, from_branch, pull_request, github_repository, username, template_y
sys.exit(1)


# nf-core rocrate
@nf_core_cli.command("rocrate")
@click.argument(
"pipeline_dir",
type=click.Path(exists=True),
default=Path.cwd(),
required=True,
metavar="<pipeline directory>",
)
@click.option(
"-j",
"--json",
default="ro-crate-metadata.json",
type=str,
help="Path to save RO Crate metadata json",
)
@click.option("-z", "--zip", type=str, help="Path to save RO Crate zip")
def rocrate(pipeline_dir, json, zip):
"""
Make an Research Object Crate
"""
import nf_core.rocrate

if json is None and zip is None:
log.error("Either --json or --zip must be specified")
sys.exit(1)
pipeline_dir = Path(pipeline_dir)
rocrate_obj = nf_core.rocrate.RoCrate(pipeline_dir)
rocrate_obj.create_ro_crate(pipeline_dir, metadata_fn=json, zip_fn=zip)


# Main script is being run - launch the CLI
if __name__ == "__main__":
run_nf_core()
11 changes: 7 additions & 4 deletions nf_core/components/components_command.py
Original file line number Diff line number Diff line change
Expand Up @@ -246,10 +246,13 @@ def check_patch_paths(self, patch_path: Path, module_name: str) -> None:
modules_json = ModulesJson(self.dir)
modules_json.load()
if modules_json.has_git_url_and_modules():
modules_json.modules_json["repos"][self.modules_repo.remote_url]["modules"][
self.modules_repo.repo_path
][module_name]["patch"] = str(patch_path.relative_to(Path(self.dir).resolve()))
modules_json.dump()
if modules_json.modules_json is not None:
modules_json.modules_json["repos"][self.modules_repo.remote_url]["modules"][
self.modules_repo.repo_path
][module_name]["patch"] = str(patch_path.relative_to(Path(self.dir).resolve()))
modules_json.dump()
else:
log.error("Could not update modules.json file.")

def check_if_in_include_stmts(self, component_path: str) -> Dict[str, List[Dict[str, Union[int, str]]]]:
"""
Expand Down
9 changes: 5 additions & 4 deletions nf_core/components/info.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import logging
import os
from pathlib import Path
from typing import Union

import questionary
import yaml
Expand Down Expand Up @@ -65,7 +66,7 @@ def __init__(
no_pull=False,
):
super().__init__(component_type, pipeline_dir, remote_url, branch, no_pull)
self.meta = None
self.meta = {}
self.local_path = None
self.remote_location = None
self.local = None
Expand Down Expand Up @@ -162,7 +163,7 @@ def get_component_info(self):

return self.generate_component_info_help()

def get_local_yaml(self):
def get_local_yaml(self) -> dict:
"""Attempt to get the meta.yml file from a locally installed module/subworkflow.

Returns:
Expand Down Expand Up @@ -201,9 +202,9 @@ def get_local_yaml(self):
return yaml.safe_load(fh)
log.debug(f"{self.component_type[:-1].title()} '{self.component}' meta.yml not found locally")

return None
return {}

def get_remote_yaml(self):
def get_remote_yaml(self) -> Union[dict, bool]:
"""Attempt to get the meta.yml file from a remote repo.

Returns:
Expand Down
17 changes: 10 additions & 7 deletions nf_core/modules/modules_json.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def __init__(self, pipeline_dir):
self.modules_dir = Path(self.dir, "modules")
self.subworkflows_dir = Path(self.dir, "subworkflows")
self.modules_json_path = Path(self.dir, "modules.json")
self.modules_json: Union(dict, None) = None
self.modules_json: Union[dict, None] = None
self.pipeline_modules = None
self.pipeline_subworkflows = None
self.pipeline_components = None
Expand Down Expand Up @@ -1056,12 +1056,15 @@ def dump(self, run_prettier: bool = False):
Sort the modules.json, and write it to file
"""
# Sort the modules.json
self.modules_json["repos"] = nf_core.utils.sort_dictionary(self.modules_json["repos"])
if run_prettier:
dump_json_with_prettier(self.modules_json_path, self.modules_json)
else:
with open(self.modules_json_path, "w") as fh:
json.dump(self.modules_json, fh, indent=4)
if self.modules_json is None:
self.load()
if self.modules_json is not None:
self.modules_json["repos"] = nf_core.utils.sort_dictionary(self.modules_json["repos"])
if run_prettier:
dump_json_with_prettier(self.modules_json_path, self.modules_json)
else:
with open(self.modules_json_path, "w") as fh:
json.dump(self.modules_json, fh, indent=4)

def resolve_missing_installation(self, missing_installation, component_type):
missing_but_in_mod_json = [
Expand Down
Loading
Loading