Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to run both SQL and RS migrations #170

Merged
merged 10 commits into from
Sep 5, 2021
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,5 @@ jobs:
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: nightly
override: true
- run: cd refinery && cargo rustdoc --all-features -- -D broken_intra_doc_links
toolchain: stable
- run: cd refinery && cargo rustdoc --all-features -- -D rustdoc::broken_intra_doc_links
13 changes: 8 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
### Unreleased (X 2021)
- Removal of `include_migration_mods` macro (Instead use `embed_migrations` macro, and there is no need to have `mod.rs`) [#154](https://github.com/rust-db/refinery/pull/154)

### v0.6.0 (July 2021)
- Update mysql to 0.21, [#164](https://github.com/rust-db/refinery/pull/164)
- Update mysql_async to 0.28, [#164](https://github.com/rust-db/refinery/pull/164)
Expand Down Expand Up @@ -25,7 +28,7 @@
- Warn when migration file name is malformed [#130](https://github.com/rust-db/refinery/pull/130)
- Add `Unversioned` migration type, [#128](https://github.com/rust-db/refinery/pull/128)
- Add `get_migrations` method to runner to allow inspecting gathered migrations, [#120](https://github.com/rust-db/refinery/pull/120)
- Add support for 'postgresl' url schema prefix, [#107](https://github.com/rust-db/refinery/pull/107)
- Add support for 'postgresql' url schema prefix, [#107](https://github.com/rust-db/refinery/pull/107)
- Add lib option to load config from a string, [1#13](https://github.com/rust-db/refinery/pull/113)
- Add lib and cli option to load config from env var, [#103](https://github.com/rust-db/refinery/pull/103)

Expand All @@ -43,7 +46,7 @@
- Rename Config.get_db_type to Config.db_type, [#95](https://github.com/rust-db/refinery/pull/95)
- Deprecate migrate_from_config and migrate_from_config_async, instead impl Migrate for Config, [#94](https://github.com/rust-db/refinery/pull/94)
- Update Runner.run and Runner.run_async return signature, Result<(), Error> -> Result<Report, Error> where report contains applied Migration's, [#92](https://github.com/rust-db/refinery/pull/92)
- Deprecate AppliedMigration, merge it's functionality into Migration, [#91](https://github.com/rust-db/refinery/pull/91)
- Deprecate AppliedMigration, merge its functionality into Migration, [#91](https://github.com/rust-db/refinery/pull/91)
- Add Runner.get_applied_migrations_async method, [#90](https://github.com/rust-db/refinery/pull/90)
- Add Runner.get_applied_migrations method, [#90](https://github.com/rust-db/refinery/pull/90)
- Add Runner.get_last_applied_migration_async method, [#90](https://github.com/rust-db/refinery/pull/90)
Expand All @@ -55,15 +58,15 @@

### v0.2.1 (February 2020)

- **Bufixes**:
- **Bugfixes**:
- Update cfg-if to 0.1.10 to fix backtrace bug [#66](https://github.com/rust-db/refinery/pull/66)

### v0.2.0 (December 2019)

- **Features**:

- Add `tokio-postgres` driver support [#10](https://github.com/rust-db/refinery/pull/19).
- Add `mysql_async` driver suport [#22](https://github.com/rust-db/refinery/pull/19).
- Add `mysql_async` driver support [#22](https://github.com/rust-db/refinery/pull/19).
- Add `migrate_from_config` function
- Add `migrate_from_config_async` function
- Update postgres to version 0.17 [#32](https://github.com/rust-db/refinery/pull/32)
Expand All @@ -72,7 +75,7 @@
- **Bugfixes**:
- allow multiple statements in migration files [#10](https://github.com/rust-db/refinery/issues/21)
- when building refinery_cli with default features, build with rusqlite bundled libsqlite3 [#33](https://github.com/rust-db/refinery/issues/21)
- rename ConnectionError to just Connection as it is a variant for Error enum, and add its source as as source [#36](https://github.com/rust-db/refinery/issues/36)
- rename ConnectionError to just Connection as it is a variant for Error enum, and add its source as source [#36](https://github.com/rust-db/refinery/issues/36)

- **Dependencies**:
- update rusqlite dependency, 0.18 -> 0.21 [#26](https://github.com/rust-db/refinery/issues/26)
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ If you are using a driver that is not yet supported, namely [`SQLx`](https://git
- Migrations can be defined in .sql files or Rust modules that must have a function called `migration` that returns a [`String`](https://doc.rust-lang.org/std/string/struct.String.html).
- Migrations can be strictly versioned by prefixing the file with `V` or not strictly versioned by prefixing the file with `U`.
- Migrations, both .sql files and Rust modules must be named in the format `[U|V]{1}__{2}.sql` or `[U|V]{1}__{2}.rs`, where `{1}` represents the migration version and `{2}` the name.
- Migrations can be run either by embedding them in your Rust code with `embed_migrations` and `include_migration_mods` macros, or via `refinery_cli`.
- Migrations can be run either by embedding them in your Rust code with `embed_migrations` macro, or via `refinery_cli`.

### Example
```rust,no_run
Expand All @@ -55,12 +55,12 @@ This is because you can be sure the next migration being run is _always_ going t

With Unversioned migrations there is more flexibility in the order that the migrations can be created and deployed.
If developer 1 creates a PR with a migration today `U11__update_cars_table.sql`, but it is reviewed for a week.
Meanwhile developer 2 creates a PR with migration `U12__create_model_tags.sql` that is much simpler and gets merged and deployed immediately.
Meanwhile, developer 2 creates a PR with migration `U12__create_model_tags.sql` that is much simpler and gets merged and deployed immediately.
This would stop developer 1's migration from ever running if you were using Versioned migrations because the next migration would need to be > 12.

## Implementation details

refinery works by creating a table that keeps all the applied migrations' versions and their metadata. When you [run](https://docs.rs/refinery/latest/refinery/struct.Runner.html#method.run) the migrations `Runner`, refinery compares the applied migrations with the the ones to be applied, checking for [divergent](https://docs.rs/refinery/latest/refinery/struct.Runner.html#method.set_abort_divergent) and [missing](https://docs.rs/refinery/latest/refinery/struct.Runner.html#method.set_abort_missing) and executing unapplied migrations.\
refinery works by creating a table that keeps all the applied migrations' versions and their metadata. When you [run](https://docs.rs/refinery/latest/refinery/struct.Runner.html#method.run) the migrations `Runner`, refinery compares the applied migrations with the ones to be applied, checking for [divergent](https://docs.rs/refinery/latest/refinery/struct.Runner.html#method.set_abort_divergent) and [missing](https://docs.rs/refinery/latest/refinery/struct.Runner.html#method.set_abort_missing) and executing unapplied migrations.\
By default, refinery runs each migration in a single transaction. Alternatively, you can also configure refinery to wrap the entire execution of all migrations in a single transaction by setting [set_grouped](https://docs.rs/refinery/latest/refinery/struct.Runner.html#method.set_grouped) to true.

### Rollback
Expand All @@ -79,7 +79,7 @@ For Rusqlite, the best way to run migrations in an async context is to run them

## Contributing

:balloon: Thanks for your help improving the project!
:balloon: Thanks for your help to improve the project!
No contribution is too small and all contributions are valued, feel free to open Issues and submit Pull Requests.

## License
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ pub fn migration() -> String {
});

m.make::<Sqlite>()
}
}
9 changes: 0 additions & 9 deletions examples/modules/main.rs

This file was deleted.

13 changes: 0 additions & 13 deletions examples/modules/migrations/V1__initial.rs

This file was deleted.

12 changes: 0 additions & 12 deletions examples/modules/migrations/V2__add_cars_table.rs

This file was deleted.

5 changes: 0 additions & 5 deletions examples/modules/migrations/mod.rs

This file was deleted.

5 changes: 2 additions & 3 deletions refinery/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,9 @@ Currently, [`Postgres`](https://crates.io/crates/postgres), [`Rusqlite`](https:/

- Migrations can be defined in .sql files or Rust modules that must have a function called `migration()` that returns a [`std::string::String`]
- Migrations, both .sql files and Rust modules must be named in the format `V{1}__{2}.rs ` where `{1}` represents the migration version and `{2}` the name.
- Migrations can be run either by embedding them on your Rust code with [`embed_migrations!`] and [`include_migration_mods!`] macros, or via `refinery_cli`.
- Migrations can be run either by embedding them on your Rust code with [`embed_migrations!`] macro, or via `refinery_cli`.

[`embed_migrations!`]: macro.embed_migrations.html
[`include_migration_mods!`]: macro.include_migration_mods.html

### Example
```rust,ignore
Expand All @@ -36,4 +35,4 @@ pub use refinery_core::config;
pub use refinery_core::{error, Error, Migration, Report, Runner, Target};
#[doc(hidden)]
pub use refinery_core::{AsyncMigrate, Migrate};
pub use refinery_macros::{embed_migrations, include_migration_mods};
pub use refinery_macros::embed_migrations;
2 changes: 2 additions & 0 deletions refinery/tests/migrations/V3__add_brand_to_cars_table.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
ALTER TABLE cars
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the addition of this sql migration on the mod migrations dir on purpose? i.e. to test a dir with both sql and mod migrations, I think since we are now merging the macro do embed migrations it probably makes sense to change the tests to work with both, what do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did it on purpose, to test both sql and mod. Also agree tests need a revisit!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok yeah so since the tests are passing and work, we can do it on a subsequent PR if you feel like doing it, and we can merge this one before

ADD COLUMN brand VARCHAR(255) NULL;
13 changes: 0 additions & 13 deletions refinery/tests/mod_migrations/V3__add_brand_to_cars_table.rs

This file was deleted.

1 change: 0 additions & 1 deletion refinery/tests/mod_migrations/mod.rs

This file was deleted.

Loading