Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move database initialization overhead into an init function #1782

Open
carolynvs opened this issue Sep 28, 2021 · 1 comment
Open

Move database initialization overhead into an init function #1782

carolynvs opened this issue Sep 28, 2021 · 1 comment
Labels
2 - 🍕 Pizza should be eaten daily performance 🏎💨 Tasks that improve the performance of Porter

Comments

@carolynvs
Copy link
Member

carolynvs commented Sep 28, 2021

There are activities that we run on every CLI command, that are not zero cost:

  • Check the schema of the current database and determining if a migration is needed
  • Ensuring indices are created on the current database

We could keep a file in PORTER_HOME that indicates if these tasks have been performed for the current storage configuration (hashed so that if the config changes, we repeat the check), and lets us only do this setup once per storage backend. This would save us 4 db calls per CLI command, which depending on the connection speed is ~1-2 seconds.

# ~/.porter/cache/init.json
storage:
  CONNECTION_HASH:
    schema: # insert the schema from the database
    indices: # hash of indices
# maybe check once an hour or something, a TTL on the check essentially

This change should be made on the v1 branch.

@carolynvs carolynvs added the 2 - 🍕 Pizza should be eaten daily label Sep 28, 2021
carolynvs added a commit to carolynvs/porter that referenced this issue Sep 28, 2021
* Consolidate creating indices into just 3 calls. We can further refine
this in getporter#1782, when we make initializing the db a single event.
* Do not ping on connect. This does mean that the first query will be
slower as it will establish the connection. So let's be aware of that
and not assume the first call just has horrible performance.

I am still looking into why the first call that establishes the
connection has bad performance.

Signed-off-by: Carolyn Van Slyck <[email protected]>
carolynvs added a commit to carolynvs/porter that referenced this issue Oct 1, 2021
* Consolidate creating indices into just 3 calls. We can further refine
this in getporter#1782, when we make initializing the db a single event.
* Do not ping on connect. This does mean that the first query will be
slower as it will establish the connection. So let's be aware of that
and not assume the first call just has horrible performance.

I am still looking into why the first call that establishes the
connection has bad performance.

Signed-off-by: Carolyn Van Slyck <[email protected]>
@carolynvs carolynvs added this to the 1.0 milestone Apr 7, 2022
@carolynvs
Copy link
Member Author

Don't forget to repeat the cosmosdb performance tests to verify that this brings performance back to acceptable levels.

@carolynvs carolynvs changed the title Move command overhead into an init function Move database initialization overhead into an init function Apr 28, 2022
@carolynvs carolynvs added performance 🏎💨 Tasks that improve the performance of Porter and removed performance 🏎💨 Tasks that improve the performance of Porter labels Apr 28, 2022
@carolynvs carolynvs removed this from the 1.0 milestone Apr 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2 - 🍕 Pizza should be eaten daily performance 🏎💨 Tasks that improve the performance of Porter
Projects
No open projects
Status: No status
Development

No branches or pull requests

2 participants