Start running Appraisals in the build matrix #2524
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nickcharlton
force-pushed
the
nc-add-appraisals-to-build-matrix
branch
from
46b411f to
2ffa316
Compare
nickcharlton
force-pushed
the
nc-add-appraisals-to-build-matrix
branch
from
2ffa316 to
c487843
Compare
This goes further with splitting out different jobs, by listing the Appraisals in the matrix. The hope is that this will provide better feedback when parts of the build fails, as if we've got something like a flakey test, it should run against every combination. In addition to this, it extracts a "composite" action which allows for some reuse between different jobs. It's not possible to reuse the services section, unfortunately (YAML anchors aren't supported, which would be the ideal solution). https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs https://docs.github.com/en/actions/creating-actions/creating-a-composite-action actions/runner#1182
nickcharlton
force-pushed
the
nc-add-appraisals-to-build-matrix
branch
from
dafb717 to
9e04caf
Compare
nickcharlton
added a commit
that referenced
this pull request
Nov 21, 2024
With Rails 8, the minimum version of Ruby becomes 3.2. Unless we drop support for prior Ruby versions (and also Rails versions), the way in which we were approaching CI can't work. Maintaining support for older Rails is not difficult because of Administrate itself, but how we approach testing so it feels unreasonable to drop support (which if we didn't test against it, we would effectively be doing) because it made CI harder to do. And we _can_ make it work, so we will. Previously, we ran the setup for the project as usual, and then ran setup for the Appraisals. This served two functions: 1. Make Appraisal available in the bundle, 2. Setup the database for running tests against However, in the CI matrix, we support multiple different versions of Ruby (3.0 and up to 3.3). As we try and add Rails 8.0, this means we can't run `bundle install` on those versions of Ruby below 3.2 as it's a dependency constraint on Rails itself. From experiments, we also can't: 1. Install Appraisal outside of the bundle, because it references gems in a way that Bundler is there to solve, 2. Load newer `schema.rb` files in older versions of Rails. Rails 7 introduced versioned schemas, and future schema versions can't be run with older versions of Rails. Instead, what we _can_ do is run the project setup inside a container configured to use the database in the CI environment, then manually do what Appraisal is doing to avoid issues bundling. This container will always use the current development version of Ruby (3.2.2, at the time of this commit) taken from the `.ruby-version` file, in which we run just enough to setup the database and ensure the later tests can run against it. It's unlikely that Administrate would do something which wasn't supported at the database level, but not impossible. However these tests would catch that (from experience: Rails falls back to a string on unknown types, so it's even less likely to fail). We also need to remove the Ruby version specification from the generated Appraisal gemfiles, as otherwise these won't install. Whilst they seemed to be valid outside of this way of running things, here, they are not. We just use `sed` to remove it in place, as it can be discarded later and removing it would cause problems with Heroku deployment of the demo app. Finally, we inline the previously extracted Action from #2524. The intent was to reduce duplication, but the order of operations are now so different it's not worth it and it would only be used in one place. Extracted from #2705. Re-inline extracted Action
nickcharlton
added a commit
that referenced
this pull request
Nov 21, 2024
With Rails 8, the minimum version of Ruby becomes 3.2. Unless we drop support for prior Ruby versions (and also Rails versions), the way in which we were approaching CI can't work. Maintaining support for older Rails is not difficult because of Administrate itself, but how we approach testing so it feels unreasonable to drop support (which if we didn't test against it, we would effectively be doing) because it made CI harder to do. And we _can_ make it work, so we will. Previously, we ran the setup for the project as usual, and then ran setup for the Appraisals. This served two functions: 1. Make Appraisal available in the bundle, 2. Setup the database for running tests against However, in the CI matrix, we support multiple different versions of Ruby (3.0 and up to 3.3). As we try and add Rails 8.0, this means we can't run `bundle install` on those versions of Ruby below 3.2 as it's a dependency constraint on Rails itself. From experiments, we also can't: 1. Install Appraisal outside of the bundle, because it references gems in a way that Bundler is there to solve, 2. Load newer `schema.rb` files in older versions of Rails. Rails 7 introduced versioned schemas, and future schema versions can't be run with older versions of Rails. Instead, what we _can_ do is run the project setup inside a container configured to use the database in the CI environment, then manually do what Appraisal is doing to avoid issues bundling. This container will always use the current development version of Ruby (3.2.2, at the time of this commit) taken from the `.ruby-version` file, in which we run just enough to setup the database and ensure the later tests can run against it. It's unlikely that Administrate would do something which wasn't supported at the database level, but not impossible. However these tests would catch that (from experience: Rails falls back to a string on unknown types, so it's even less likely to fail). We also need to remove the Ruby version specification from the generated Appraisal gemfiles, as otherwise these won't install. Whilst they seemed to be valid outside of this way of running things, here, they are not. We just use `sed` to remove it in place, as it can be discarded later and removing it would cause problems with Heroku deployment of the demo app. Finally, we inline the previously extracted Action from #2524. The intent was to reduce duplication, but the order of operations are now so different it's not worth it and it would only be used in one place. Extracted from #2705. Re-inline extracted Action
nickcharlton
added a commit
that referenced
this pull request
Nov 21, 2024
With Rails 8, the minimum version of Ruby becomes 3.2. Unless we drop support for prior Ruby versions (and also Rails versions), the way in which we were approaching CI can't work. Maintaining support for older Rails is not difficult because of Administrate itself, but how we approach testing so it feels unreasonable to drop support (which if we didn't test against it, we would effectively be doing) because it made CI harder to do. And we _can_ make it work, so we will. Previously, we ran the setup for the project as usual, and then ran setup for the Appraisals. This served two functions: 1. Make Appraisal available in the bundle, 2. Setup the database for running tests against However, in the CI matrix, we support multiple different versions of Ruby (3.0 and up to 3.3). As we try and add Rails 8.0, this means we can't run `bundle install` on those versions of Ruby below 3.2 as it's a dependency constraint on Rails itself. From experiments, we also can't: 1. Install Appraisal outside of the bundle, because it references gems in a way that Bundler is there to solve, 2. Load newer `schema.rb` files in older versions of Rails. Rails 7 introduced versioned schemas, and future schema versions can't be run with older versions of Rails. Instead, what we _can_ do is run the project setup inside a container configured to use the database in the CI environment, then manually do what Appraisal is doing to avoid issues bundling. This container will always use the current development version of Ruby (3.2.2, at the time of this commit) taken from the `.ruby-version` file, in which we run just enough to setup the database and ensure the later tests can run against it. It's unlikely that Administrate would do something which wasn't supported at the database level, but not impossible. However these tests would catch that (from experience: Rails falls back to a string on unknown types, so it's even less likely to fail). We also need to remove the Ruby version specification from the generated Appraisal gemfiles, as otherwise these won't install. Whilst they seemed to be valid outside of this way of running things, here, they are not. We just use `sed` to remove it in place, as it can be discarded later and removing it would cause problems with Heroku deployment of the demo app. Finally, we inline the previously extracted Action from #2524. The intent was to reduce duplication, but the order of operations are now so different it's not worth it and it would only be used in one place. Extracted from #2705.
nickcharlton
added a commit
that referenced
this pull request
Nov 21, 2024
With Rails 8, the minimum version of Ruby becomes 3.2. Unless we drop support for prior Ruby versions (and also Rails versions), the way in which we were approaching CI can't work. Maintaining support for older Rails is not difficult because of Administrate itself, but how we approach testing so it feels unreasonable to drop support (which if we didn't test against it, we would effectively be doing) because it made CI harder to do. And we _can_ make it work, so we will. Previously, we ran the setup for the project as usual, and then ran setup for the Appraisals. This served two functions: 1. Make Appraisal available in the bundle, 2. Setup the database for running tests against However, in the CI matrix, we support multiple different versions of Ruby (3.0 and up to 3.3). As we try and add Rails 8.0, this means we can't run `bundle install` on those versions of Ruby below 3.2 as it's a dependency constraint on Rails itself. From experiments, we also can't: 1. Install Appraisal outside of the bundle, because it references gems in a way that Bundler is there to solve, 2. Load newer `schema.rb` files in older versions of Rails. Rails 7 introduced versioned schemas, and future schema versions can't be run with older versions of Rails. Instead, what we _can_ do is run the project setup inside a container configured to use the database in the CI environment, then manually do what Appraisal is doing to avoid issues bundling. This container will always use the current development version of Ruby (3.2.2, at the time of this commit) taken from the `.ruby-version` file, in which we run just enough to setup the database and ensure the later tests can run against it. It's unlikely that Administrate would do something which wasn't supported at the database level, but not impossible. However these tests would catch that (from experience: Rails falls back to a string on unknown types, so it's even less likely to fail). We also need to remove the Ruby version specification from the generated Appraisal gemfiles, as otherwise these won't install. Whilst they seemed to be valid outside of this way of running things, here, they are not. We just use `sed` to remove it in place, as it can be discarded later and removing it would cause problems with Heroku deployment of the demo app. Finally, we inline the previously extracted Action from #2524. The intent was to reduce duplication, but the order of operations are now so different it's not worth it and it would only be used in one place. Extracted from #2705.
nickcharlton
added a commit
that referenced
this pull request
Nov 22, 2024
With Rails 8, the minimum version of Ruby becomes 3.2. Unless we drop support for prior Ruby versions (and also Rails versions), the way in which we were approaching CI can't work. Maintaining support for older Rails is not difficult because of Administrate itself, but how we approach testing so it feels unreasonable to drop support (which if we didn't test against it, we would effectively be doing) because it made CI harder to do. And we _can_ make it work, so we will. Previously, we ran the setup for the project as usual, and then ran setup for the Appraisals. This served two functions: 1. Make Appraisal available in the bundle, 2. Setup the database for running tests against However, in the CI matrix, we support multiple different versions of Ruby (3.0 and up to 3.3). As we try and add Rails 8.0, this means we can't run `bundle install` on those versions of Ruby below 3.2 as it's a dependency constraint on Rails itself. From experiments, we also can't: 1. Install Appraisal outside of the bundle, because it references gems in a way that Bundler is there to solve, 2. Load newer `schema.rb` files in older versions of Rails. Rails 7 introduced versioned schemas, and future schema versions can't be run with older versions of Rails. Instead, what we _can_ do is run the project setup inside a container configured to use the database in the CI environment, then manually do what Appraisal is doing to avoid issues bundling. This container will always use the current development version of Ruby (3.2.2, at the time of this commit) taken from the `.ruby-version` file, in which we run just enough to setup the database and ensure the later tests can run against it. It's unlikely that Administrate would do something which wasn't supported at the database level, but not impossible. However these tests would catch that (from experience: Rails falls back to a string on unknown types, so it's even less likely to fail). We also need to remove the Ruby version specification from the generated Appraisal gemfiles, as otherwise these won't install. Whilst they seemed to be valid outside of this way of running things, here, they are not. We just use `sed` to remove it in place, as it can be discarded later and removing it would cause problems with Heroku deployment of the demo app. Finally, we inline the previously extracted Action from #2524. The intent was to reduce duplication, but the order of operations are now so different it's not worth it and it would only be used in one place. Extracted from #2705.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This goes further with splitting out different jobs, by listing the Appraisals in the matrix. The hope is that this will provide better feedback when parts of the build fails, as if we've got something like a flakey test, it should run against every combination.
In addition to this, it extracts a "composite" action which allows for some reuse between different jobs. It's not possible to reuse the services section, unfortunately (YAML anchors aren't supported, which would be the ideal solution).
https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs
https://docs.github.com/en/actions/creating-actions/creating-a-composite-action
actions/runner#1182