Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(worker): Prevent timeout on large validator output by waiting on process.communicate instead of exit #3235

Merged
merged 1 commit into from
Nov 25, 2024

Conversation

nellh
Copy link
Contributor

@nellh nellh commented Nov 25, 2024

This simplifies running the schema validator, we wait on process.communicate instead of process exit before reading the stdout buffer. This avoids a deadlock where deno would wait for the buffer to be consumed before writing the rest of the output.

Fixes #3233

Copy link

codecov bot commented Nov 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 45.66%. Comparing base (4a5bf01) to head (ea5407e).
Report is 7 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #3235   +/-   ##
=======================================
  Coverage   45.66%   45.66%           
=======================================
  Files         595      595           
  Lines       38013    38008    -5     
  Branches     1167     1167           
=======================================
- Hits        17357    17355    -2     
+ Misses      20459    20456    -3     
  Partials      197      197           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@nellh nellh merged commit 2c696df into master Nov 25, 2024
14 of 15 checks passed
@nellh nellh deleted the 3233-validator-timeout-on-large-output branch November 25, 2024 22:11
@alessandra-rampinini
Copy link

alessandra-rampinini commented Nov 26, 2024

Hi! The dataset for v1.0.2 at https://openneuro.org/datasets/ds005613/versions/1.0.2 still shows validation pending, while it shows 'valid' if I visualize the draft.
Draft (but lots of warnings not present before):
image
v1.0.2
image

Plus there are additional issues:

  1. it shows a huge difference in size while I only updated the readme
    v1.0.2
    image
    v1.0.1
    image

  2. it looks invalid from my dataset list:
    image

EDIT:
3) randomly shows this upon opening v.1.0.2
image

I only corrected a typo in the readme for the publication between v.1.0.1 and v.1.0.2, so I'm wondering what is happening. Please feel free to move this conversation to a new issue if it does not belong to the original request anymore.
best,
alessandra

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Validation pending ds005613 after updating readme file
2 participants