Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove storage cache layer (#324) #326

Merged
merged 2 commits into from
Jan 26, 2017

Conversation

codingllama
Copy link
Contributor

The following changes have been made:

  • Open *sql.DB outside of storage (and reuse the same DB for all storage objects)
  • Test refactors (storage/mysql):
    • Moved DB-handling methods to storage_test (which contains TestMain)
    • Added a *sql.DB parameter to methods that access storage
    • Open DB connection only once for all tests
    • Cleaned DB at the beginning of every test

The changes above render the storage caching layer obsolete.

}

// cleanTestDB deletes all the entries in the database. Only use this with a test database
func cleanTestDB(db *sql.DB) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using a variant of the test specific new db logic in testonly/integration which creates a separate database for each test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not merged yet, added a TODO.

}
}

func createTestDB(db *sql.DB) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using the temporary tree admin api in storage/mysql/log_admin.go

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, I'll add a TODO for now.

}

// prepareTestTreeDB removes all database contents for the specified log id so tests run in a predictable environment. For obvious reasons this should only be allowed to run against test databases. This method panics if any of the deletions fails to make sure tests can't inadvertently succeed.
func prepareTestTreeDB(db *sql.DB, treeID int64, t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider deleting the existing test database and creating a new one for test freshness.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a TODO.

}

// prepareTestLogDB removes all database contents for the specified log id so tests run in a predictable environment. For obvious reasons this should only be allowed to run against test databases. This method panics if any of the deletions fails to make sure tests can't inadvertently succeed.
func prepareTestLogDB(db *sql.DB, logID logIDAndTest, t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using the tree admin api

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, I'll add a TODO for now.

@gdbelvin gdbelvin mentioned this pull request Jan 26, 2017
Alan Parra added 2 commits January 26, 2017 12:12
The following changes have been made:
* Open *sql.DB outside of storage (and reuse the same DB for all storage
  objects)
* Test refactors (storage/mysql):
  * Moved DB-handling methods to storage_test (which contains TestMain)
  * Added a *sql.DB parameter to methods that access storage
  * Open DB connection only once for all tests
  * Cleaned DB at the beginning of every test

The changes above render the storage caching layer obsolete.
@gdbelvin
Copy link
Contributor

These changes look great.

@codingllama
Copy link
Contributor Author

Thanks, merging!

@codingllama codingllama merged commit e95788b into google:master Jan 26, 2017
@daviddrysdale
Copy link
Contributor

If I run the log integration test there seem to be an awful lot of calls to defaultRegistry.GetLogStorage() -- is this expected?

(If I add a line to that function with log.Infof("Creating new storage for log: %d", treeID) and turn on --alsologtostderr for the trillian_log_server invocation in integration/log_integration_test.sh, then running the test hits it ~850 times.)

@gdbelvin
Copy link
Contributor

gdbelvin commented Jan 31, 2017 via email

@daviddrysdale
Copy link
Contributor

OK, thanks for checking.

@codingllama codingllama deleted the storage-cache branch February 1, 2017 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants