Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Rails 6.1 #1556

Merged
merged 8 commits into from
Feb 22, 2021
Merged

Update to Rails 6.1 #1556

merged 8 commits into from
Feb 22, 2021

Conversation

david-a-wheeler
Copy link
Collaborator

This commit begins an update from Rails 6.0.X -> 6.1.X.
This requires a large number of simultaneous library updates
(some took a while to identify).

This does not complete the update.

The biggest problem is the many error reports of this form:

Error:
ProjectsControllerTest#test_should_fail_to_create_project_with_duplicate_repo:
ActiveRecord::StatementInvalid: PG::UndefinedColumn:
ERROR: column users.email does not exist

It's true that there is no users.email column in the database,
but that's because it's a virtual column that is supposed to be
managed by the attr_encrypted gem.
Since it's not being handled, it appears that this gem does not
work with ActiveRecord 6.1. I went to check out its status, and they
are looking for new maintainers:
attr-encrypted/attr_encrypted#379
All options are not the desired ones here.

In addition, there are at least two kinds of deprecation warnings
which will need to be addressed (probably many times):

  • DEPRECATION WARNING: action_view.raise_on_missing_translations is deprecated and will be removed in Rails 6.2. Set i18n.raise_on_missing_translations instead. Note that this new setting also affects how missing translations are handled in controllers. (called from call at /home/dwheeler/best-practices-badge/config/initializers/canonical_trailing_slash.rb:30)

  • DEPRECATION WARNING: Rendering actions with '.' in the name is deprecated: static_pages/error_404.html.erb (called from error_404 at /home/dwheeler/best-practices-badge/app/controllers/static_pages_controller.rb:46)

Signed-off-by: David A. Wheeler [email protected]

This commit *begins* an update from Rails 6.0.X -> 6.1.X.
This requires a large number of simultaneous library updates
(some took a while to identify).

This does *not* complete the update.

The biggest problem is the many error reports of this form:

> Error:
> ProjectsControllerTest#test_should_fail_to_create_project_with_duplicate_repo:
> ActiveRecord::StatementInvalid: PG::UndefinedColumn:
> ERROR:  column users.email does not exist

It's true that there is no users.email column in the database,
but that's because it's a virtual column that is *supposed* to be
managed by the `attr_encrypted` gem.
Since it's not being handled, it *appears* that this gem does not
work with ActiveRecord 6.1. I went to check out its status, and they
are looking for new maintainers:
attr-encrypted/attr_encrypted#379
All options are not the desired ones here.

In addition, there are at least two kinds of deprecation warnings
which will need to be addressed (probably many times):

* DEPRECATION WARNING: action_view.raise_on_missing_translations is deprecated and will be removed in Rails 6.2. Set i18n.raise_on_missing_translations instead. Note that this new setting also affects how missing translations are handled in controllers. (called from call at /home/dwheeler/best-practices-badge/config/initializers/canonical_trailing_slash.rb:30)

* DEPRECATION WARNING: Rendering actions with '.' in the name is deprecated: static_pages/error_404.html.erb (called from error_404 at /home/dwheeler/best-practices-badge/app/controllers/static_pages_controller.rb:46)

Signed-off-by: David A. Wheeler <[email protected]>
@david-a-wheeler david-a-wheeler changed the title Update to Rails 6.1 (partially done) Update to Rails 6.1 (partially done) - DO NOT MERGE Feb 10, 2021
@david-a-wheeler
Copy link
Collaborator Author

Do not merge yet. I just wanted to show public progress. There are some nontrivial problems that need to be addressed.

@david-a-wheeler
Copy link
Collaborator Author

@jdossett (and anyone else) - good news, it's a little awkward, but a lot of the Rails 6.0->6.1 conversion can be done by simultaneously upgrading a lot of other libraries.

The deprecation warnings are a pain, but should be straightforward to fix.

HOWEVER: the fact that the attr_encrypted gem doesn't seem to work any more is a big potential problem :-(. It's a single parameter, so I might be able to just hand-define a getter & setter for "email" (which is really what attr_encrypted does). But it's more work that we didn't have to hand-code before. Sigh.

Update gem blind_index (0.3.4->2.2.0) with a tweak to
how it's called, and tweak tests so that the tests at least run
and pass.

There are many deprecation warnings (that's next).

Signed-off-by: David A. Wheeler <[email protected]>
VCR sometimes gets called in ways that end up invoking the
chromedriver checks, even when we're not doing system tests.
Move the configuration so that we *NEVER* capture version tests
that have nothing to do with the software under test (SUT).

Signed-off-by: David A. Wheeler <[email protected]>
@codecov
Copy link

codecov bot commented Feb 12, 2021

Codecov Report

Merging #1556 (223697d) into master (83e03a5) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff            @@
##            master     #1556   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           53        53           
  Lines         1926      1926           
=========================================
  Hits          1926      1926           
Impacted Files Coverage Δ
app/controllers/badge_static_controller.rb 100.00% <ø> (ø)
app/controllers/projects_controller.rb 100.00% <ø> (ø)
app/controllers/sessions_controller.rb 100.00% <ø> (ø)
app/controllers/static_pages_controller.rb 100.00% <ø> (ø)
app/models/user.rb 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 83e03a5...223697d. Read the comment docs.

@david-a-wheeler
Copy link
Collaborator Author

@jdossett - HALLELUJAH!

This looks like a working transition to Rails 6.1.

Comments welcome.

@david-a-wheeler david-a-wheeler changed the title Update to Rails 6.1 (partially done) - DO NOT MERGE Update to Rails 6.1 Feb 12, 2021
This updates gem web-console, an in-browser debugger.
Insert <% console %> or console into the source code to use it.
This gem has been transferred to be part of the "rails" project.

Note that we only include this gem during development,
*not* in production. This gem enables running arbitrary code,
which would be a security problem if it was included in production.
But we do not call it from production code, and we don't include
the gem in production mode, preventing a security issue.

Signed-off-by: David A. Wheeler <[email protected]>
@david-a-wheeler
Copy link
Collaborator Author

david-a-wheeler commented Feb 12, 2021

Clarification: the gem attr_encrypted works just fine. The problem was that I needed to update the gem blind_index, but the error messages misled me to blame the wrong gem. All's okay now.

@jdossett
Copy link
Contributor

Wow @david-a-wheeler, great work!

@david-a-wheeler
Copy link
Collaborator Author

@jdossett - thanks!! What a pain, but success feels sweet. Any comments before merging?

@david-a-wheeler david-a-wheeler changed the title Update to Rails 6.1 Update to Rails 6.1 - DO NOT MERGE YET Feb 15, 2021
@david-a-wheeler
Copy link
Collaborator Author

I have found a problem. The updated gem blind_index stores email hashes differently, so while it works with new data (and thus automated tests work), it won't find old email addresses (and thus won't allow local logins for existing accounts).

This will need to be fixed before merging. One solution is to regenerate all the hashes (e.g., as part of a migration). However, there are over 4,000 such accounts, so doing that all at once during a migration will take a little time. That may be the easy way though.

The sessions controller uses the standard method names,
but it might not be obvious what they mean in the context
of the sessions controller. This adds comments to clarify that.

Signed-off-by: David A. Wheeler <[email protected]>
The update to the blind_index gem requires a number
of changes no matter what.

This commit switches the application to the newer
blind_index naming conventions and algorithm (argon2id),
requiring a migration to do it and changes to the
test fixture for user data.

This commit was more work than expected.

Calculating the bidx values doesn't work correctly
from the fixtures within the ERB (the calls *run* but don't
produce the right values, presumably because it's not fully set up
at that point). This was resolved by inserting the calculated values
into the fixtures, which also makes the tests more independent
of the system being tested.
One of the older test values was subtly wrong (.com vs. .org
email address), which created a mysterious hard-to-track failure
(we prevent future versions of this problem
via a new test that sanity-checks all test values).

The migration had to be tweaked several times to work with
real production data. We need to not change the user updated_at
by setting touch to false (the user-visible data isn't changing),
we need not validate (some entries may not be acceptable any more
but we still need to migrate them as they are), and we need to
skip cases where encrypted_email is nil (in those cases
we have no data to decrypt).

Signed-off-by: David A. Wheeler <[email protected]>
@david-a-wheeler
Copy link
Collaborator Author

Regenerating all the hashes doesn't take long at all, it's about a minute.

However, there were a crazy number of little details that needed to be worked out before I had a finished solution. I think I've got it.

@david-a-wheeler david-a-wheeler changed the title Update to Rails 6.1 - DO NOT MERGE YET Update to Rails 6.1 Feb 20, 2021
@david-a-wheeler
Copy link
Collaborator Author

@jdossett or anyone else, please let me know if you have any concerns about merging this. I think it's all well. The tests all pass. I did a trial conversion of the real database & that seemed to work.

@david-a-wheeler
Copy link
Collaborator Author

I hear no complaints, and I see no reason to complain. I'm finally merging it in!

@david-a-wheeler david-a-wheeler merged commit 5326fad into master Feb 22, 2021
@david-a-wheeler david-a-wheeler deleted the rails_6_1 branch February 22, 2021 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants