Avoid unintentionally canceling the scheduled crate publishing job #13088

joao-paulo-parity · 2023-01-06T13:03:38Z

Since publish-crates and publish-crates-manual share the resource group crates-publishing, any instance of publish-crates-manual (i.e. even instances created but not run) cancels a running instance of publish-crates, as demonstrated by https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2212179. A workaround for that unintended interaction is to avoid creating manual instances of publish-crates-manual and instead require pipelines to be triggered manually by checking $CI_PIPELINE_SOURCE == "web". See #13088 (comment).

close https://github.com/paritytech/release-engineering/issues/146

because publish-crates and publish-crates-manual share the resource group "crates-publishing", any instance of publish-crates-manual cancels a running instance of publish-crates, as demonstrated by https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2212179 a workaround for that unintended interaction is to avoid creating instances of publish-crates-manual and instead require pipelines to be triggered manually by checking $CI_JOB_MANUAL == "true"

add comments

joao-paulo-parity · 2023-01-06T15:40:06Z

I think my changes up to 533e1ff are not enough for avoiding the pipeline cancelation. On a broader look of https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines/236214 I noticed that not only publish-crates-locally but many other jobs, such as quick-benchmarks, were also cancelled. I think this was due to the interaction between interruptible: true and the "Auto-cancel redundant pipelines" GitLab CI settings; i.e. it might not be related to resource_group as (wrongly) deduced in #13088 (comment).

So I came up with a different workaround. @alvicsam @oleg-plakida please take another look.

alvicsam

Why not add interruptible: false for the required job? The option can be added per job

joao-paulo-parity · 2023-01-10T10:05:23Z

Why not add interruptible: false for the required job? The option can be added per job

@alvicsam The publish-crates job is at the publish stage. That stage might not be reached in case some jobs are canceled, right? e.g. if the test stage's jobs gets canceled, for instance.

Furthermore I think it's relevant for the whole pipeline to be run in order to ensure that the code is working correctly before publishing it. I don't want to skip or cancel the test or check stages' jobs for instance.

In light of that, if I'd go for the individual interruptible: false usage, I think I'd have to add it not only to the job in question but all the ones it needs - directly or indirectly. Quoting https://docs.gitlab.com/ee/ci/yaml/#interruptible

You can’t cancel subsequent jobs after a job with interruptible: false starts.

So considering all of this, it seems simpler and less cumbersome to set interruptible: false for everything.

alvicsam

Okay, I was hoping for an easier solution. A few nits and we can merge it.

.gitlab-ci.yml

Co-authored-by: Alexander Samusev <[email protected]>

joao-paulo-parity · 2023-01-10T12:01:36Z

bot merge

…13088) * avoid unintentionally canceling the scheduled crate publishing job because publish-crates and publish-crates-manual share the resource group "crates-publishing", any instance of publish-crates-manual cancels a running instance of publish-crates, as demonstrated by https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2212179 a workaround for that unintended interaction is to avoid creating instances of publish-crates-manual and instead require pipelines to be triggered manually by checking $CI_JOB_MANUAL == "true" * check manual pipelines by $CI_PIPELINE_SOURCE instead of $CI_JOB_MANUAL * make crate-publishing pipelines uninterruptible * use conditional includes to work around interruptible limitations * organize comments * remove interruptible from common pipeline * wip: check include * wip: check include * fix include * fix include * fix include * fix yaml * fix yaml * remove shared common-pipeline * wip: retry common-pipeline * move .default-template to .gitlab-ci.yml * fix the pipeline add comments * fix default-pipeline.yml * revert publish-crates-manual to when: manual * move "needs:" back to publish-crates * avoid manual repetition * improve previous commit * try to avoid manual repetition * fix indentation * minor adjustments * move defaults to top of .gitlab-ci.yml * fix positioning on default in the diff * comments * indentation * Apply suggestions from code review Co-authored-by: Alexander Samusev <[email protected]> Co-authored-by: Alexander Samusev <[email protected]>

…aritytech#13088) * avoid unintentionally canceling the scheduled crate publishing job because publish-crates and publish-crates-manual share the resource group "crates-publishing", any instance of publish-crates-manual cancels a running instance of publish-crates, as demonstrated by https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2212179 a workaround for that unintended interaction is to avoid creating instances of publish-crates-manual and instead require pipelines to be triggered manually by checking $CI_JOB_MANUAL == "true" * check manual pipelines by $CI_PIPELINE_SOURCE instead of $CI_JOB_MANUAL * make crate-publishing pipelines uninterruptible * use conditional includes to work around interruptible limitations * organize comments * remove interruptible from common pipeline * wip: check include * wip: check include * fix include * fix include * fix include * fix yaml * fix yaml * remove shared common-pipeline * wip: retry common-pipeline * move .default-template to .gitlab-ci.yml * fix the pipeline add comments * fix default-pipeline.yml * revert publish-crates-manual to when: manual * move "needs:" back to publish-crates * avoid manual repetition * improve previous commit * try to avoid manual repetition * fix indentation * minor adjustments * move defaults to top of .gitlab-ci.yml * fix positioning on default in the diff * comments * indentation * Apply suggestions from code review Co-authored-by: Alexander Samusev <[email protected]> Co-authored-by: Alexander Samusev <[email protected]>

joao-paulo-parity requested a review from a team as a code owner January 6, 2023 13:03

alvicsam approved these changes Jan 6, 2023

View reviewed changes

paritytech-ci requested a review from a team January 6, 2023 13:12

check manual pipelines by $CI_PIPELINE_SOURCE instead of $CI_JOB_MANUAL

533e1ff

mateo-moon approved these changes Jan 6, 2023

View reviewed changes

joao-paulo-parity added 17 commits January 6, 2023 11:26

make crate-publishing pipelines uninterruptible

ae6493f

use conditional includes to work around interruptible limitations

a7a522c

organize comments

70ef904

remove interruptible from common pipeline

f16de79

wip: check include

8d5ea86

wip: check include

370b140

fix include

0bc5958

fix include

618c509

fix include

6e4421f

fix yaml

716fa11

fix yaml

261d47c

remove shared common-pipeline

6ef0120

wip: retry common-pipeline

7553643

move .default-template to .gitlab-ci.yml

211c9ef

fix the pipeline

e791c05

add comments

fix default-pipeline.yml

74d128e

revert publish-crates-manual to when: manual

2a68e42

joao-paulo-parity added 2 commits January 6, 2023 12:41

move "needs:" back to publish-crates

cf20d13

Merge branch 'master' of github.com:paritytech/substrate into fix

c57cb15

try to avoid manual repetition

0dac70d

alvicsam self-requested a review January 9, 2023 10:21

joao-paulo-parity added 4 commits January 9, 2023 07:24

fix indentation

9c02881

minor adjustments

294c092

move defaults to top of .gitlab-ci.yml

c5952d6

fix positioning on default in the diff

3f2909a

alvicsam suggested changes Jan 10, 2023

View reviewed changes

paritytech-ci requested a review from a team January 10, 2023 09:19

joao-paulo-parity requested a review from alvicsam January 10, 2023 10:05

alvicsam suggested changes Jan 10, 2023

View reviewed changes

.gitlab-ci.yml Outdated Show resolved Hide resolved

.gitlab-ci.yml Show resolved Hide resolved

.gitlab-ci.yml Show resolved Hide resolved

paritytech-ci requested a review from a team January 10, 2023 10:40

joao-paulo-parity added 2 commits January 10, 2023 08:19

comments

479da91

indentation

08d899d

joao-paulo-parity requested a review from alvicsam January 10, 2023 11:30

alvicsam suggested changes Jan 10, 2023

View reviewed changes

.gitlab-ci.yml Outdated Show resolved Hide resolved

.gitlab-ci.yml Outdated Show resolved Hide resolved

paritytech-ci requested a review from a team January 10, 2023 11:33

Apply suggestions from code review

c969998

Co-authored-by: Alexander Samusev <[email protected]>

alvicsam approved these changes Jan 10, 2023

View reviewed changes

mateo-moon approved these changes Jan 10, 2023

View reviewed changes

paritytech-processbot bot merged commit 4e38342 into paritytech:master Jan 10, 2023

joao-paulo-parity deleted the fix branch January 10, 2023 12:01

coderobe mentioned this pull request Jan 11, 2023

ci: mark update-weights non-interruptible paritytech/polkadot#6539

Merged

joao-paulo-parity mentioned this pull request Jan 16, 2023

More improvements for the crate publishing pipeline #13153

Merged

aurexav mentioned this pull request Feb 8, 2023

To polkadot-v0.9.37 darwinia-network/darwinia-2.0#267

Closed

kacperzuk-neti mentioned this pull request Feb 14, 2023

Polkadot v0.9.37 (pallet-nfts) liberland/liberland_substrate#244

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid unintentionally canceling the scheduled crate publishing job #13088

Avoid unintentionally canceling the scheduled crate publishing job #13088

joao-paulo-parity commented Jan 6, 2023 •

edited

Loading

joao-paulo-parity commented Jan 6, 2023

alvicsam left a comment

joao-paulo-parity commented Jan 10, 2023

alvicsam left a comment

joao-paulo-parity commented Jan 10, 2023

Avoid unintentionally canceling the scheduled crate publishing job #13088

Avoid unintentionally canceling the scheduled crate publishing job #13088

Conversation

joao-paulo-parity commented Jan 6, 2023 • edited Loading

joao-paulo-parity commented Jan 6, 2023

alvicsam left a comment

Choose a reason for hiding this comment

joao-paulo-parity commented Jan 10, 2023

alvicsam left a comment

Choose a reason for hiding this comment

joao-paulo-parity commented Jan 10, 2023

joao-paulo-parity commented Jan 6, 2023 •

edited

Loading