-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate CI to ENF runners & new platform framework #8
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Should submodule check be successful? But it should not be a blocking issue. |
linh2931
approved these changes
Aug 16, 2022
submodule check will work again as-is one repo is public, but let me see if I can easily fix it for private repo |
heifner
added a commit
that referenced
this pull request
Apr 29, 2024
heifner
added a commit
that referenced
this pull request
Apr 29, 2024
heifner
added a commit
that referenced
this pull request
Apr 29, 2024
heifner
added a commit
that referenced
this pull request
Apr 29, 2024
heifner
added a commit
that referenced
this pull request
Apr 29, 2024
heifner
added a commit
that referenced
this pull request
Apr 29, 2024
heifner
added a commit
that referenced
this pull request
Apr 29, 2024
heifner
added a commit
that referenced
this pull request
Apr 29, 2024
IF: Missing vote logging and Prometheus logging of votes
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Migrate CI to the new high performance ENF runners with flexible platform framework. This improves build time and build time consistency while at the same time eliminating complexity and fragility such as
ccache
. The platform flexibility gives us easier feedback on builds over a wider range of platforms. Currently builds are performed on Ubuntu 18, 20, and 22. Pinned & ARM builds will need to be added at a later time but prototyping has shown this approach can accommodate pinned & ARM builds with only a slight addition in current complexity.Some other minor changes are included such as the ability to manually launch a build, and general reduction in the amount of error prone copy pasta around what long running tests to run via automatically populating a matrix from what
ctest
reports.Unfortunately there are some major outstanding problems. The Ubuntu 18 tests consistently fail due to a timing fault in
test_trx_full
which is not understood – sometimes the producer fails to produce a block on the scale of a dozen seconds. The Ubuntu 22 tests fail due to boost filesystem erroneously assuming it cancopy_file_range()
any files, but the way GitHub Runner sets up filesystem mounts in a containerized job causescopy_file_range()
to run afoul of anEXDEV
error. This is fixed in later boost filesystem versions but that doesn’t help us since we want these unpinned builds to be representative of what users would encounter.So at the moment 18 & 22 tests are disabled.
The original intention was to fan out NP tests through a matrix of GitHub runners (since these NP tests don’t need high performance), however, at least in the current private repo, GitHub aggressively throttles GitHub runner creation when we go and try to make a matrix of 100 jobs. The current ENF runner framework only is capable of launching high core count VMs, so we can’t efficiently use ENF runners for these jobs. Thus, sadly, all NP tests continue to be run in serial taking 30 minutes or so to complete.
Some extra cruft is present due to the repo being private:
discover-platforms-action
should instead use the shared repo. A follow on PR will be made to clean up those bits once public.