Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a logic to execute and render js expressions in CWL Schema #5

Closed
nsyzrantsev opened this issue Jan 5, 2025 · 0 comments · Fixed by #6
Closed

Implement a logic to execute and render js expressions in CWL Schema #5

nsyzrantsev opened this issue Jan 5, 2025 · 0 comments · Fixed by #6
Assignees
Labels
component: core enhancement New feature or request

Comments

@nsyzrantsev
Copy link
Contributor

nsyzrantsev commented Jan 5, 2025

Description

Implement a logic to zefiro-cwl-parser crate to evaluate JavaScript expressions and populates schema fields containing expressions in the formats $(...), ${...}, or | ${...}. For this purpose, the deno_core crate will be utilized.

https://www.commonwl.org/v1.2/CommandLineTool.html#Expressions_(Optional)

Inputs:

  • fastq: s3://bucket/sample/file.fastq.gz (size: 1024 MB or 1024*1024*1024 bytes)
  • fastq2: s3://bucket/sample/file.fastq.gz (size: 512 MB or 512*1024*1024 bytes)
...
inputs:
  - id: fastq
    type: File
    inputBinding:
      prefix: --fastq
  - id: fastq2
    type: File
    inputBinding:
      prefix: --fastq2
...
requirements:
  - class: ResourceRequirement
    coresMin: 8
    outdirMin: |
      ${
        var in_files_size = (inputs.fastq.size + inputs.fastq2.size) / (1024 * 1024);
        return 2 * in_files_size;
      }
  - class: ToolTimeLimit
    timelimit: $(60 * 60 * 8)
  ...

Outputs:

...
inputs:
  - id: fastq
    type: File
    inputBinding:
      prefix: --fastq
  - id: fastq2
    type: File
    inputBinding:
      prefix: --fastq2
  ...
requirements:
  - class: ResourceRequirement
    coresMin: 8
    outdirMin: 3072
  - class: ToolTimeLimit
    timelimit: 28800
  ...

Questions:

  1. How can expressions in the formats $(...), ${...}, or | ${...} be accurately matched in a schema? Should the matching rely on input paths within values and the presence of ${ or $( symbols at the start?
  2. How should input file sizes be evaluated correctly? How it will work for remote files from AWS, GCP, Azure?
  3. Which cloud platforms will be supported in the implementation? Or will the current realization work exclusively with local files?
  4. What js data structure of inputs will be used? And when basename, nameroot, nameext etc. should be generated?
"inputs": {
  "fastq": {
      "class": "File",
      "location": "/path/to/input.txt",
      "basename": "input.txt",
      "nameroot": "input",
      "nameext": ".txt",
      "checksum": "0912349fasf2143",
      "size": 536870912,
  }
}
@nsyzrantsev nsyzrantsev self-assigned this Jan 5, 2025
@nsyzrantsev nsyzrantsev converted this from a draft issue Jan 5, 2025
@nsyzrantsev nsyzrantsev added enhancement New feature or request component: core labels Jan 5, 2025
@nsyzrantsev nsyzrantsev changed the title Implement a crate to execute and render js expressions in CWL Schema Implement a logic to execute and render js expressions in CWL Schema Jan 6, 2025
@nsyzrantsev nsyzrantsev linked a pull request Jan 10, 2025 that will close this issue
@nsyzrantsev nsyzrantsev moved this from Todo to Done in Zefiro Project Roadmap Jan 10, 2025
@nsyzrantsev nsyzrantsev closed this as completed by moving to Done in Zefiro Project Roadmap Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: core enhancement New feature or request
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant