fix(db-handler): wait for secrets when they aren't available #813
+307
−32
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of terminating, the connection enters a waiting state, and responds to checkout attempts with:
This helps in a few situations:
Avoids restart loop when a pool has no client connections and stale credentials, and poolboy wants to start a db handler to keep it warm. Instead of trying to start a DbHandler that fails immediately, the DbHandler will be waiting until an user connects with fresh credentials, and only then tries to connect to the upstream db.
Avoids multiple wrong password attempts when credentials change with a pool running with multiple open connections. E.g.: assume we have 100 client connections, 10 db connections, and the user changes the password. Eventually, an 11th connection is needed. Starting the DbHandler with old credentials will fail with a bad password. Before, every attempt to create this 11th connection would result in an error. With 100 existing client connections, we could try to auth to the upstream db up to 100 times, if no new client connections came up to refresh our cache. This could lead supavisor to get banned by the upstream db. Now the 11th connection will hog a slot in the pool and refuse checkouts, so the pool continues existing, degraded, until a new connection comes up, without impacting connectivity to the upstream db.