upcoming Rails 6 multi-database

[edk]

I know this is probably a bit soon, but after watching the talk by Eileen (https://confreaks.tv/videos/railsconf2018-keynote-the-future-of-rails-6-scalable-by-default), I started wondering how the upcoming changes to multi-database support will work (or not work) with the way apartment is currently architected.

Is anyone tracking the in-progress work and perhaps joining in, to guide it to make apartment happily co-exist with rails 6?

[ericraio]

any thoughts?

[fuzzygroup]

I too would love to know this. I'm also curious about Apartment gem and Rails 5 since I can find no mention of Rails 5 and Apartment.

[styrmis]

@fuzzygroup It looks like Rails 5 support was added in v1.2.0, with Rails 5.2 support added in v2.2.0.

[ericraio]

Would it be safe to migrate an apartment app to Rails 6?

[jean-francois-labbe]

Hi,

On Rails 6, tests are failling due to parallelize(workers: :number_of_processors).

I can run my tests tests by commenting the parallelize(workers: :number_of_processors), but we're now missing one big rails 6 feature.

At first sight it seems that apartment is working with Rails 6.
@fuzzygroup I can confirm that apartment is working with Rails 5, currently using it with a rails 5.2

[mikecmpbll]

only tracked it from a distance. i was under the impression the multidatabase stuff was targeted towards clustering stuff but i'm going to watch the talk tonight and get a better view :). thanks for raising it.

[mikecmpbll]

two main areas of note from the talk:

• parallel tests
  • i don't use apartment in my test suite, so not sure of the implications of this but there's some discussion on Add rails 6 to test matrix #603 which i'll look over shortly! should be able to get it to work without much issue.
  • the threaded paralleliser won't work with connection-based adapters because of the wellknown threadsafety issues there
• multi-db support
  • the basis of this is the "3-tier" support in database.yml
  • designed for when using different databases for different models
  • lots of changes and additions to the db rake tasks in order to support this setup
  • apartment will need changes to also support this configuration too
  • presumably, any model which uses a non-primary connection which is not in excluded_models should be tenanted in the same way as models with the primary connection

[ericraio]

two main areas of note from the talk:

• parallel tests

  • i don't use apartment in my test suite, so not sure of the implications of this but there's some discussion on Add rails 6 to test matrix #603 which i'll look over shortly! should be able to get it to work without much issue.
  • the threaded paralleliser won't work with connection-based adapters because of the wellknown threadsafety issues there

• multi-db support

  • the basis of this is the "3-tier" support in database.yml
  • designed for when using different databases for different models
  • lots of changes and additions to the db rake tasks in order to support this setup
  • apartment will need changes to also support this configuration too
  • presumably, any model which uses a non-primary connection which is not in excluded_models should be tenanted in the same way as models with the primary connection

This could be a big win.

Currently, we have the public schema where the purpose is to be a meta schema across each tenant schema.

From a code perspective, this looks clean but when looking at the database, the public schema is cluttered with unwanted tables. Kinda like how when you look at a STI model, and you end up with a bunch of unwanted fields across each model.

Now if Apartment can take advantage of the multi-migration support and reference accordingly I believe it would be a cleaner approach. rails/rails#36205 (comment)

[ericraio]

@mikecmpbll Any thoughts to above? ☝️

[mikecmpbll]

@ericraio sorry eric. good point, I kinda missed that angle. because of the ability to target migrations to only certain "databases", you could indeed separate your public schema migrations. I guess the setup would be something like;

production:
  primary:
    <<: *defaults
  shared:
    <<: *defaults
    schema_search_path: public
    migrations_paths: 'db/shared_migrate'

Apartment.excluded_models = ["Foo", "Bar"]

then target your excluded model migrations at the shared connection by placing them at db/shared_migrate/*. i don't see any obvious conflicts with apartment apart from some possible issues with the way we commandeer the db rake tasks. i look forward to hearing people's experiences.

[ericraio]

@mikecmpbll I really like this configuration!

[ericraio]

@ericraio sorry eric. good point, I kinda missed that angle. because of the ability to target migrations to only certain "databases", you could indeed separate your public schema migrations. I guess the setup would be something like;

production:
  primary:
    <<: *defaults
  shared:
    <<: *defaults
    schema_search_path: public
    migrations_paths: 'db/shared_migrate'

Apartment.excluded_models = ["Foo", "Bar"]

then target your excluded model migrations at the shared connection by placing them at db/shared_migrate/*. i don't see any obvious conflicts with apartment apart from some possible issues with the way we commandeer the db rake tasks. i look forward to hearing people's experiences.

@mikecmpbll sorry kinda misread what you wrote. Are you saying that without changing Apartment, this would work out of the box with the above configuration or would we need to make some adjustments?

[ericraio]

@mikecmpbll quick follow up :)

[mikecmpbll]

basically saying I don't really know, but it looks like it should work out of the box :).

[rpbaltazar]

Is there any sort of roadmap or feature list that needs to be completed before being able to release a version compatible with rails 6?

I understand that master is theoretically compatible at the moment, but having some sort of publicly released version, as well as features that are supported and what not is sort of helpful.

If there is a list of things that need to be done I could try to help with them.

[jmschneider]

If the current version works with Rails 6 but just doesn't integrate tightly with the latest features, would it be possible to get a patch version bump to reflect that it works with Rails 6?

[rpbaltazar]

We are planning to start working on this.
Here are the steps I've done so far:

1. forked the current apartment to to https://github.com/rails-on-services/apartment. My plan for this is to track all issues that we identify related with rails 6.
2. released an alpha gem to test the gem in our project

In my opinion, if apartment features support rails 6, then we should just release a version for rails 6 and start adding the support for rails multi db on another version and that's what I plan to do.

If anyone wants to help out in this process please feel free to reach out.

[rpbaltazar]

We are planning to start working on this.
Here are the steps I've done so far:

1. forked the current apartment to to https://github.com/rails-on-services/apartment. My plan for this is to track all issues that we identify related with rails 6.

2. released an alpha gem to test the gem in our project

In my opinion, if apartment features support rails 6, then we should just release a version for rails 6 and start adding the support for rails multi db on another version and that's what I plan to do.

Some more status updates:

The branch I'm working on and from which https://github.com/rails-on-services/apartment/tree/rails-6-testing-release has been setup in travis as well so I can have reporting on the status of things.

The gem i'm currently using is: https://rubygems.org/gems/ros-apartment which has no changes yet from the apartment development branch.

[rpbaltazar]

Final update and I'll stop spamming everyone in this thread.
For those of you interested in giving it a try, I've released the gem mentioned in my previous comment and I'm starting to update my application to use it. So far I have not encountered any problems, but you never know.

https://github.com/rails-on-services/apartment

I've opened the issues so if you find any bugs while trying it, please feel free to log the issues.

[rabi-nsox]

Thanks to rpbaltazar for the ros-apartment gem.
We upgraded our apartment based app to rails6 with no issues.
We'll make sure we file any bugs we find.