Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.14 sync broke again #1071

Closed
ewels opened this issue May 11, 2021 · 0 comments · Fixed by #1072
Closed

1.14 sync broke again #1071

ewels opened this issue May 11, 2021 · 0 comments · Fixed by #1072
Assignees
Labels
bug Something isn't working high-priority

Comments

@ewels
Copy link
Member

ewels commented May 11, 2021

We got quite far this time:

  • ✅ Local sync was good
  • ✅ Push was fine & branch was created on the repo
  • ✅ The PR was created
  • ✅ The abuse mechanism was tolerated with retries
    • Still didn't find the Retry-After header, but this time we had backup code that just waited a random number of seconds and that worked beautifully.
    • See community forum post about the missing header

Then when we tried to get the URL for the new PR that had been created with self.pr_url = self.gh_pr_returned_data["html_url"] we got: TypeError: byte indices must be integers or slices, not str

This affected all pipeline syncs and the crash meant that the code to close existing PRs never ran.

Example output from the GitHub Actions run (viralrecon):

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 1.14



INFO     Clearing all items from the cache                           base.py:108
INFO     Pipeline directory:                                          sync.py:87
         /home/runner/work/tools/tools/nf-core/viralrecon                       
INFO     Using branch 'dev' to fetch workflow variables               sync.py:89
INFO     Will attempt to automatically create a pull request          sync.py:91
INFO     Original pipeline repository branch is 'dev'                sync.py:143
INFO     Deleting all files in 'TEMPLATE' branch                     sync.py:199
INFO     Making a new template pipeline using pipeline variables     sync.py:217
INFO     Committed changes to 'TEMPLATE' branch                      sync.py:243
INFO     Pushing TEMPLATE branch to remote: 'viralrecon'             sync.py:253
INFO     Checking out merge base branch                              sync.py:281
         'nf-core-template-merge-1.14'                                          
INFO     Pushing 'nf-core-template-merge-1.14' branch to remote      sync.py:289
INFO     Submitting a pull request via the GitHub API                sync.py:302
WARNING  Got 403 code - probably the abuse protection. Trying again  sync.py:356
         after 23 seconds..                                                     
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /opt/hostedtoolcache/Python/3.8.10/x64/bin/nf-core:8 in <module>             │
│                                                                              │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])     │
│ ❱ 8 │   sys.exit(run_nf_core())                                              │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/nf_core/_ │
│ _main__.py:58 in run_nf_core                                                 │
│                                                                              │
│    57 │   # Lanch the click cli                                              │
│ ❱  58 │   nf_core_cli()                                                      │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/click/cor │
│ e.py:829 in __call__                                                         │
│                                                                              │
│    828 │   │   """Alias for :meth:`main`."""                                 │
│ ❱  829 │   │   return self.main(*args, **kwargs)                             │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/click/cor │
│ e.py:782 in main                                                             │
│                                                                              │
│    781 │   │   │   │   with self.make_context(prog_name, args, **extra) as c │
│ ❱  782 │   │   │   │   │   rv = self.invoke(ctx)                             │
│    783 │   │   │   │   │   if not standalone_mode:                           │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/click/cor │
│ e.py:1259 in invoke                                                          │
│                                                                              │
│   1258 │   │   │   │   with sub_ctx:                                         │
│ ❱ 1259 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/click/cor │
│ e.py:1066 in invoke                                                          │
│                                                                              │
│   1065 │   │   if self.callback is not None:                                 │
│ ❱ 1066 │   │   │   return ctx.invoke(self.callback, **ctx.params)            │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/click/cor │
│ e.py:610 in invoke                                                           │
│                                                                              │
│    609 │   │   │   with ctx:                                                 │
│ ❱  610 │   │   │   │   return callback(*args, **kwargs)                      │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/nf_core/_ │
│ _main__.py:670 in sync                                                       │
│                                                                              │
│   669 │   try:                                                               │
│ ❱ 670 │   │   sync_obj.sync()                                                │
│   671 │   except (nf_core.sync.SyncException, nf_core.sync.PullRequestExcept │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/nf_core/s │
│ ync.py:114 in sync                                                           │
│                                                                              │
│   113 │   │   │   │   self.push_merge_branch()                               │
│ ❱ 114 │   │   │   │   self.make_pull_request()                               │
│   115 │   │   │   │   self.close_open_template_merge_prs()                   │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/nf_core/s │
│ ync.py:343 in make_pull_request                                              │
│                                                                              │
│   342 │   │   │   if r.status_code == 201:                                   │
│ ❱ 343 │   │   │   │   self.pr_url = self.gh_pr_returned_data["html_url"]     │
│   344 │   │   │   │   log.debug(f"GitHub API PR                              │
│       worked:\n{returned_data_prettyprint}\n\n{r_headers_pp}")               │
╰──────────────────────────────────────────────────────────────────────────────╯
TypeError: byte indices must be integers or slices, not str
Error: Process completed with exit code 1.
@ewels ewels added the bug Something isn't working label May 11, 2021
@ewels ewels self-assigned this May 11, 2021
@ewels ewels mentioned this issue May 11, 2021
4 tasks
@ewels ewels linked a pull request May 11, 2021 that will close this issue
4 tasks
@ewels ewels closed this as completed May 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working high-priority
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant