Skip to content

Conversation

ashanbrown
Copy link

To ensure timely cleanup of resources, dispose of any engines after using them.

This need surfaced because psyopg3 reports a warning later if a connection if gc'd, making it desirable to dispose of the engine (and its connection pool) even if the command fails.

This pattern is already in place for database_exists, so I'm just suggesting we extend it for create_engine and drop_database.

Copy link
Collaborator

@kurtmckee kurtmckee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashanbrown Thanks for submitting this.

Please add a test that demonstrates that this issue both exists and is fixed by this change. Thanks!

Comment on lines 565 to 568
try:
if dialect_name == 'postgresql':
if not template:
template = 'template1'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kicks off a try/except block that is larger than it should be.

I recommend modifying this so that the conditionals exclusively determine what the text variable should be.

text = ''
if dialect_name == 'postgresql':
    if not template:
        template = 'template1'
    text = "CREATE DATABASE {} CHARACTER SET = '{}'".format(
        quote(conn, database), encoding
    )
elif dialect_name == 'mysql':
    ...
    text = ...
elif dialect_name == 'sqlite' and database != ':memory:':
    if database:
        text = 'CREATE TABLE DB(id int); DROP TABLE DB;'
else:
    text = f'CREATE DATABASE {quote(conn, database)}'

and then reduce the try/except block to something like this, which accommodates the possibility that text is an empty string.

if text:
    try:
        with engine.begin() as conn:
            conn.execute(sa.text(text))
    finally:
        engine.dispose()

(It's possible that engine.dispose() still needs to be run, so please check whether the if text: conditional should actually be inside the try block.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least one of the examples (sqlite), requires multiple executing multiple text values, so this could possibly get a little messy. If we just had a context manager provide an engine that is always disposed, would that resolve your concern? Then we could also clean up duplicate code between create_database and drop_database.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I addressed that by putting both commands into the same line. I don't anticipate it being messy, but could be wrong.

elif dialect_name == 'sqlite' and database != ':memory:':
    if database:
        text = 'CREATE TABLE DB(id int); DROP TABLE DB;'

pool = engine.pool

# a disposed engine should not have the same pool
assert engine.pool is not pool
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also mock the dispose method if this doesn't seem sufficient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants