Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Silence failing tests on Windows and add Windows build step #7293

Merged
merged 9 commits into from
Jan 26, 2022

Conversation

tagniam
Copy link
Contributor

@tagniam tagniam commented Dec 1, 2021

What this PR does / why we need it:
While working on the bang/filter command feature, I've noticed that quite a few tests are failing only on Windows. This PR proposes we temporarily silence those tests on Windows and document them with TODO comments, which I hope should be helpful towards fixing #4844 (and #4079).

Additionally, I propose we add a Windows build step to our Github Actions workflow to encourage cross-platform compatibility for future PRs. However, this may affect this repo's Github Actions minutes limit - jobs that run on Windows consume minutes at twice the rate of Linux-based jobs. This multiplier means that this would probably cause the minutes consumption to triple. I'm not sure what the minutes limit is for VSCodeVim nor if we currently run near that limit, so if this would cause problems, I can omit this change. Lemme know your thoughts!

Which issue(s) this PR fixes
N/A, but progresses towards fixing #4844 and #4079.

@github-actions github-actions bot force-pushed the build-step-for-windows branch from dcf6113 to 02c15c5 Compare December 1, 2021 04:45
@tagniam tagniam force-pushed the build-step-for-windows branch from 02c15c5 to dc968eb Compare December 1, 2021 04:50
@github-actions github-actions bot force-pushed the build-step-for-windows branch from dc968eb to 96182bf Compare December 1, 2021 04:51
@tagniam tagniam force-pushed the build-step-for-windows branch from 96182bf to c207ad9 Compare December 1, 2021 05:10
@tagniam tagniam closed this Dec 1, 2021
@tagniam tagniam reopened this Dec 1, 2021
test/jumpTracker.test.ts Outdated Show resolved Hide resolved
@J-Fields
Copy link
Member

J-Fields commented Dec 1, 2021

Thanks for the PR! I'm definitely supportive of this idea. We should do something similar for the web version as well at some point (maybe Mac as well, but it seems that uses 10x the minutes).

Any idea why this last step won't finish?
image

I'm not sure what the minutes limit is for VSCodeVim nor if we currently run near that limit

I think the free limit is 2000 minutes/month - I don't think we're anywhere near that. If this causes issues, we can always change to run on all platforms only for releases, or something like that.

@tagniam tagniam changed the base branch from master to beta_release_test December 2, 2021 03:22
@tagniam tagniam changed the base branch from beta_release_test to master December 2, 2021 03:23
@tagniam tagniam force-pushed the build-step-for-windows branch from 6e426dd to 50c7551 Compare December 2, 2021 03:24
@tagniam
Copy link
Contributor Author

tagniam commented Dec 2, 2021

I think the free limit is 2000 minutes/month - I don't think we're anywhere near that. If this causes issues, we can always change to run on all platforms only for releases, or something like that.

Sounds good!

Any idea why this last step won't finish?

Not sure, the other two steps are fine 🤔 I've tried closing/reopening the PR, force pushing and even changing the base branch. Going to try and open another PR to see if this is a fluke.

@tagniam
Copy link
Contributor Author

tagniam commented Jan 9, 2022

Sorry for the delay! From what I can tell, the last build step is forever waiting because it comes from the existing enforced status check rule - it is applying to this branch as well, but since the rule was updated and no longer exists on this branch, it's waiting forever.

You might need to merge this in first and then modify the required status checks in this repo's Settings > Branches > Require status checks to pass before merging to these new build / build (ubuntu-latest) and build / build (windows-latest) checks. Related: https://circleci.com/docs/2.0/enable-checks/#troubleshooting-github-checks-waiting-for-status-in-github


One thing I'd like to bring up is that it seems running tests on Windows is a bit flakier, sometimes the Windows build fails while the ubuntu build succeeds, in which a re-run usually fixes it. If that is a concern we can shelve the Windows build for now, or perhaps make it a non-required check for PRs. Thoughts?

@J-Fields
Copy link
Member

Sorry for the delay!

No trouble whatsoever, I haven't been active on this project at all recently.

You might need to merge this in first and then modify the required status checks

Sounds good, I'll give that a shot

One thing I'd like to bring up is that it seems running tests on Windows is a bit flakier, sometimes the Windows build fails while the ubuntu build succeeds, in which a re-run usually fixes it. If that is a concern we can shelve the Windows build for now, or perhaps make it a non-required check for PRs. Thoughts?

Yeah good point. Non-required is probably the way to go for now

@J-Fields J-Fields merged commit b7fd095 into VSCodeVim:master Jan 26, 2022
@J-Fields
Copy link
Member

Seems to have worked, thanks very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants