Skip to content

Commit

Permalink
Reformat docs only.
Browse files Browse the repository at this point in the history
  • Loading branch information
jzohrab committed Apr 10, 2016
1 parent 6e25f56 commit f3b7fbe
Show file tree
Hide file tree
Showing 3 changed files with 88 additions and 33 deletions.
36 changes: 24 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,26 @@

## Overview

A simple command-line python framework to handle automatic DB migrations and updates, with working examples provided for MySQL and postgres.
A simple command-line python framework to handle automatic DB
migrations and updates, with working examples provided for MySQL and
postgres.

This tool was written for particular requirements that existing database migration tools (eg, Redgate, DbUp, etc) don't support:
This tool was written for particular requirements that existing
database migration tools (eg, Redgate, DbUp, etc) don't support:

* cross platform (written in python, can support different database platforms. Note that the concepts here can be easily ported to another language)
* cross platform (written in python, can support different database
platforms. Note that the concepts here can be easily ported to
another language)
* database scripts are written in sensible platform-specific SQL dialect, not a DSL
* suggests an approach to database change management comprised of baseline schema, migrations, reference files, and "database code" (e.g., views, stored procedures, etc). See [Managing Database Changes](docs/managing_database_changes.md) for more detail.
* suggests an approach to database change management comprised of
baseline schema, migrations, reference files, and "database code"
(e.g., views, stored procedures, etc). See [Managing Database
Changes](docs/managing_database_changes.md) for more detail.
* supports a distributed development model

Notes on the code, including unit testing and implementing custom extensions to handle your project's migrations if necessary, are in the [code overview](docs/code_overview.md).
Notes on the code, including unit testing and implementing custom
extensions to handle your project's migrations if necessary, are in
the [code overview](docs/code_overview.md).

## Examples

Expand Down Expand Up @@ -39,12 +49,12 @@ used as-is.

### Database setup

The tests and examples manipulate actual databases running on your system.
Start the local mysql and postgres database servers before running
tests.
The tests and examples manipulate actual databases running on your
system. Start the local mysql and postgres database servers before
running tests.

The tests assume the existence of the following accounts with admin access
to the databases (to create and destroy objects, etc):
The tests assume the existence of the following accounts with admin
access to the databases (to create and destroy objects, etc):

| |MySql|Postgres|
|--- |--- |--- |
Expand All @@ -71,5 +81,7 @@ $ source venv/bin/activate
See the [code overview](./docs/code_overview.md) for notes about structure.

1. Fork the repo.
2. Create the feature, add necessary test coverage, and ensure existing tests still pass.
3. Push to your fork, and submit a GitHub pull request back to the master branch.
2. Create the feature, add necessary test coverage, and ensure
existing tests still pass.
3. Push to your fork, and submit a GitHub pull request back to the
master branch.
57 changes: 44 additions & 13 deletions docs/code_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,59 @@

## migrator module

The **migrator** module contains the migration library classes. The central **Migrator** class (Facade pattern) reads the appropriate database files from disk using an impementation of the abstract class **DatabaseSource**, and passes them to a **QueuedScriptCollection** which sorts the scripts into the correct execution order. The scripts are passed in order to a **ScriptRunner** instance, which calls an implementation of the abstract class **DatabaseHandler** as required by the target database platform. The **ScriptRunner** determines if scripts should be run or not (it creates and manages a table of previously run migrations in a tracking table in the database).
The **migrator** module contains the migration library classes. The
central **Migrator** class (Facade pattern) reads the appropriate
database files from disk using an impementation of the abstract class
**DatabaseSource**, and passes them to a **QueuedScriptCollection**
which sorts the scripts into the correct execution order. The scripts
are passed in order to a **ScriptRunner** instance, which calls an
implementation of the abstract class **DatabaseHandler** as required
by the target database platform. The **ScriptRunner** determines if
scripts should be run or not (it creates and manages a table of
previously run migrations in a tracking table in the database).

The **DatabaseSource** and **DatabaseHandler** are abstract to allow for the **Migrator** to be re-used in different projects that have different directory structures, and different database platforms.
The **DatabaseSource** and **DatabaseHandler** are abstract to allow
for the **Migrator** to be re-used in different projects that have
different directory structures, and different database platforms.

The [postgres example](../examples/README.md) illustrates these classes, and some sensible defaults.
The [postgres example](../examples/README.md) illustrates these
classes, and some sensible defaults.


## Unit and Integration Testing

Tests are in `./test`. They ran be run from project root with `make test`.

The database tests destructively test their databases (destroy and recreate, etc.).
The database tests destructively test their databases (destroy and
recreate, etc.).


## Using the Framework

As mentioned in the Overview, this tool does not handle all database migrations out-of-the-box, unless you follow the same layout given in the Postgres or MySQL examples. The main **Migrator** class handles the migrations, but it needs to understand your particular project and database platform, and it needs a wrapper that satisfies your automation requirements. You must implement some abstract classes and a wrapper/driver class (sensible defaults are also provided, which may either be used outright, or as the basis for custom code):

1. Implement abstract class **migrator.DatabaseSource**. This class pulls database migration files from disk, as required for your project's directory structure. A **DefaultDatabaseSource** is included, which can be used if your project's database files are laid out as described in that class's documentation.

2. Implement abstract class **migrator.DatabaseHandler**. This class should be implemented and tested for your database platform. The **PostgresDatabaseHandler** is provided for postgres databases.

3. Implement a wrapper class, callable from the command line, which calls the appropriate **Migrator** methods. The **Driver** class can either be reused for your project, or as sample code (note that the **Driver** class is used by the examples).

The [examples](../examples) show working examples which use the defaults. You can choose if custom coding or project structure changes would be easier to introduce into your workflow.
As mentioned in the Overview, this tool does not handle all database
migrations out-of-the-box, unless you follow the same layout given in
the Postgres or MySQL examples. The main **Migrator** class handles
the migrations, but it needs to understand your particular project and
database platform, and it needs a wrapper that satisfies your
automation requirements. You must implement some abstract classes and
a wrapper/driver class (sensible defaults are also provided, which may
either be used outright, or as the basis for custom code):

1. Implement abstract class **migrator.DatabaseSource**. This class
pulls database migration files from disk, as required for your
project's directory structure. A **DefaultDatabaseSource** is
included, which can be used if your project's database files are laid
out as described in that class's documentation.

2. Implement abstract class **migrator.DatabaseHandler**. This class
should be implemented and tested for your database platform. The
**PostgresDatabaseHandler** is provided for postgres databases.

3. Implement a wrapper class, callable from the command line, which
calls the appropriate **Migrator** methods. The **Driver** class can
either be reused for your project, or as sample code (note that the
**Driver** class is used by the examples).

The [examples](../examples) show working examples which use the
defaults. You can choose if custom coding or project structure
changes would be easier to introduce into your workflow.
28 changes: 20 additions & 8 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,17 @@ This folder contains working examples for postgres and mysql.

## Postgres

The postgres example runs the schema and migration files in the postgres_test on a Postgres database. See the postgres.py documentation for notes.
The postgres example runs the schema and migration files in the
postgres_test on a Postgres database. See the postgres.py
documentation for notes.

You'll need to configure the connection strings for your postgres database to run this example. In that directory, copy connections.ini.template, and rename it to connections.ini. Fill in the details as instructed in that file.
You'll need to configure the connection strings for your postgres
database to run this example. In that directory, copy
connections.ini.template, and rename it to connections.ini. Fill in
the details as instructed in that file.

The example is run from the command line, with command-line argument parsing. Here is the help output:
The example is run from the command line, with command-line argument
parsing. Here is the help output:

````
> python postgres.py -h
Expand All @@ -29,9 +35,11 @@ optional arguments:
-u, --update Updates database (runs migrations, code, and data)
````

Note: python.py driver sets the "default database" referred to above to "postgres_test", the database given in the connection.ini file.
Note: python.py driver sets the "default database" referred to above
to "postgres_test", the database given in the connection.ini file.

An actual run (-nsu = "create a new database, run the baseline schema on it, and then update it"):
An actual run (-nsu = "create a new database, run the baseline schema
on it, and then update it"):

````
> python postgres.py -nsu
Expand All @@ -43,17 +51,21 @@ Execute 10_vX.sql on postgres_test
Execute bootstrap_data.sql on postgres_test
````

Once migrations have been run, they are not run again, so repeating an upgrade skips the migration files (201304*.sql):
Once migrations have been run, they are not run again, so repeating an
upgrade skips the migration files (201304*.sql):

````
> python postgres.py -u
Execute 10_vX.sql on postgres_test
Execute bootstrap_data.sql on postgres_test
````

More notes on schema, migrations, code, and data are given in [Managing Database Changes](../docs/managing_database_changes.md).
More notes on schema, migrations, code, and data are given in
[Managing Database Changes](../docs/managing_database_changes.md).


## MySQL

The MySQL example is the same as the postgres example above, with "mysql" substituted for "postgres" in the various filenames and directories.
The MySQL example is the same as the postgres example above, with
"mysql" substituted for "postgres" in the various filenames and
directories.

0 comments on commit f3b7fbe

Please sign in to comment.