Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce "use database" and update gaiac usage #633

Merged
merged 7 commits into from
Apr 26, 2021
Merged

Introduce "use database" and update gaiac usage #633

merged 7 commits into from
Apr 26, 2021

Conversation

chuan
Copy link
Contributor

@chuan chuan commented Apr 22, 2021

  • Introduce the use database statement. With this, users can switch DB context in the DDL. Tables without a database name will be assumed in the current database context.
  • Update gaiac command line usage in the following ways.
    • No database creation or name inference from the DDL filename.
      • Database creation must be explicitly done via create database DDL statement.
      • Users can create multiple databases with create database in the same DDL file.
    • Users can generate headers for multiple databases at once by specifying multiple -d database in the command line.
    • When no output path is given, the DDL file basename will be used as output path.

For example, gaiac -d barn_storage -d lab -g incubator.ddl will generate barn_storage and lab headers under incubator directory.

@chuan chuan requested review from simone-gaia, senderista, LaurentiuCristofor and daxhaw and removed request for senderista April 22, 2021 18:03

#
# Generate stand alone incubator.
#
add_custom_command(
COMMENT "Compiling incubator.ddl..."
OUTPUT ${GAIA_GENERATED_SCHEMAS}/gaia_barn_storage.h
COMMAND ${GAIA_GENERATE} --destroy-db -t ${GAIA_PROD_BUILD}/db/core -o ${GAIA_GENERATED_SCHEMAS} -g ${INCUBATOR_DDL}
COMMAND ${GAIA_GENERATE} --destroy-db -t ${GAIA_PROD_BUILD}/db/core -o ${GAIA_GENERATED_SCHEMAS} -d barn_storage -g ${INCUBATOR_DDL}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the folder called "barn_storage" when the command says "incubator"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so the database for "incubator" is called "barn_storage". Using two names for this test is rather confusing. Why not have the database called incubator as well? Or just stick to barn_storage.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For prerequisites, the ddl and the database use the same name. It would be nice to come up with some consistent naming scheme to eliminate surprises about what is called which way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. But this is how this script/schema is currently written and used by many tests.

Copy link
Contributor

@LaurentiuCristofor LaurentiuCristofor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good, but I'm not sure about some file/database renames - the naming of files and databases seems inconsistent across tests/demos. We should come up with a more consistent naming scheme.

@chuan chuan requested a review from waynelwarren April 22, 2021 23:41
production/catalog/README.md Outdated Show resolved Hide resolved
@@ -21,6 +21,11 @@ void initialize_catalog()
ddl_executor_t::get();
}

void use_database(const string& name)
{
ddl_executor_t::get().switch_db_context(name);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there will be a problem if you have multiple threads using this (eg. multiple sessions)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a problem for multiple threads because ddl_executor_t is a singleton and we use unique lock when switching context.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure what you mean by multiple sessions. DB sessions are irrelevant here.

If you mean multiple processes, we always have the issue that the data structures in ddl_executor_t are not consistent across multiple processes.

Copy link
Contributor

@simone-gaia simone-gaia Apr 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have multiple threads, within the same process, each running their own session.

What happens if each of these threads to something that causes switch_db_context to be called?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They will be executed in sequence due to the use of unique lock. The db context a statement see will be the last use statement before it. Remember DDLs are not subject to txn semantics. So we don't need to guarantee a consistent view for DDLs executed in parallel. More practically speaking, gaiac will not execute DDLs in parallel.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough.

Copy link
Contributor

@simone-gaia simone-gaia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chuan chuan merged commit aa24cf2 into master Apr 26, 2021
@chuan chuan deleted the use-database branch April 26, 2021 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants