Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent usage of failed connections and improve connection management #1156

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

adamyeats
Copy link
Contributor

@adamyeats adamyeats commented Feb 7, 2025

This PR improves connection management and error handling to ensure that failed connections are not returned to the pool, preventing the use of invalid connections.

  • Ensures failed PingContext(ctx) connections are not returned to the pool. If PingContext(ctx) fails, db is no longer returned to prevent an unusable connection from being used.
  • Moves timeout creation (context.WithTimeout) closer to PingContext(ctx). Prevents premature cancellations that could occur during connection setup.
  • Ensures that the timeout only applies to the connection attempt itself.
  • Adds explicit connection pool settings for better efficiency.
    • Sets MaxOpenConns to 50.
    • Sets MaxIdleConns to 25.
  • The select check was moved back to before PingContext(ctx) to avoid connecting if ctx.Done() is already triggered. The error from PingContext(ctx) is now handled outside of the select as PingContext(ctx) is context aware.

@adamyeats adamyeats self-assigned this Feb 7, 2025
@adamyeats adamyeats requested a review from a team as a code owner February 7, 2025 16:36
@adamyeats adamyeats requested a review from alyssabull February 7, 2025 16:36
@adamyeats adamyeats changed the title Set reasonable defaults for db pool, and don't return db when PingContext returns an error Prevent usage of failed connections and improve connection management Feb 7, 2025
@adamyeats adamyeats force-pushed the pool-defaults-ping-error branch from 3d78e3b to 03c123f Compare February 7, 2025 17:11
@adamyeats adamyeats marked this pull request as draft February 7, 2025 17:14
@adamyeats adamyeats force-pushed the pool-defaults-ping-error branch 4 times, most recently from 91d4d63 to 996b1b6 Compare February 7, 2025 18:10
@adamyeats adamyeats force-pushed the pool-defaults-ping-error branch from 996b1b6 to a6b2b28 Compare February 7, 2025 18:20
@adamyeats adamyeats marked this pull request as ready for review February 7, 2025 18:20
Copy link
Contributor

@aangelisc aangelisc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM! The only suggestion I would make is to have connection lifetime, max idle connections, and max open connections configurable as we do in other data sources. It'd be nice to do that in this PR but feel free to also do it later on.

I definitely think this change should explicitly be called out in the changelog with the previous defaults vs the current defaults. I'm wondering if it constitutes a breaking change but I'm not sure if anyone would be reliant on these settings. Wdyt?

Copy link
Collaborator

@SpencerTorres SpencerTorres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always good to see this kind of async logic cleaned up 👍

I agree with @aangelisc, more configuration is nice, even if we don't write the frontend UI for it yet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In Review
Development

Successfully merging this pull request may close these issues.

3 participants