Fix to: Breaking change in 0.42.0 - multiple database sessions

[lovelace9981]

Hi, I want to contribute with this PR a fix for Issue 791, because if we don't terminate the background activity in any PostgreSQL database when we want to drop it, the operation fails due to remaining connections or activity.

I improved a bit on this comment to include a FORCE option, which is available from PostgreSQL 13, allowing a similar behavior without manually calling the pg_terminate_backend() function.

This change has been tested on PostgreSQL 9, 13, and the latest version (18).

[kurtmckee]

It seems like #795 is addressing the cause of this issue, based on what I'm briefly reading in the #791.

[lovelace9981]

Hi, thank you for your response!

I downloaded and tested the PR you mentioned, but unfortunately, it doesn’t fully fix the issue. To reproduce the problem, we need to simulate a situation where a connection remains open (not disposed nor terminated) before dropping a database.

In PostgreSQL, if we try to drop a database that still has an active connection, query, or transaction, the RDBMS does not allow the operation and raises an error.

To clarify the problem, I’ve included the traceback produced when running the drop_database test with an unfinished connection:

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f4abfe03ed0>
cursor = <cursor object at 0x7f4acc1e95d0; closed: -1>
statement = 'DROP DATABASE db_test_sqlalchemy_util', parameters = {}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f4abe51c150>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.ObjectInUse: database "db_test_sqlalchemy_util" is being accessed by other users
E       DETAIL:  There is 1 other session using the database.

To reproduce the problem in drop database I induced that problem with this code.

       # Generate a conn that isn't dropped
        engine = sa.create_engine(url, **engine_kwargs)
        conn = engine.connect()
        result = conn.execute(sa.text("SELECT version()"))
       # Trying to DROP the database
        with _create_engine(url, use_primary_db=True, **engine_kwargs) as engine:
            with engine.begin() as conn:
                text = f'DROP DATABASE {quote(conn, database)}'
                conn.execute(sa.text(text))

Thanks again

[lovelace9981]

Investigating a little bit, with this change maybe we can expand the TestDatabasePostgres to include a new testing method to reproduce the use case:

@pytest.mark.usefixtures('postgresql_dsn')
class TestDatabasePostgres(DatabaseTest):

    @pytest.fixture
    def db_name(self):
        return 'db_test_sqlalchemy_util'

    def test_template(self, postgresql_db_user, postgresql_db_password):
        dsn = 'postgresql://{}:{}@localhost/db_test_sqlalchemy_util'.format(
            postgresql_db_user,
            postgresql_db_password
        )
        with pytest.raises(sa.exc.ProgrammingError) as excinfo:
            create_database(dsn, template='my_template')
        assert ("CREATE DATABASE db_test_sqlalchemy_util ENCODING 'utf8' "
                "TEMPLATE my_template") in str(excinfo.value)

    def test_drop_database_with_conn(self, dsn):
        assert not database_exists(dsn)
        create_database(dsn)
        assert database_exists(dsn)

        engine = sa.create_engine(dsn)
        conn = engine.connect()
        result = conn.execute(sa.text("SELECT 1"))
        assert result.all()
        
        try:
            drop_database(dsn)
        except psycopg2.errors.ObjectInUse:
            pytest.fail("drop_database() raised ObjectInUse unexpectedly")

        with pytest.raises(Exception):
            conn.close()

        assert not database_exists(dsn)

Thanks again