Skip to content

fix: permission error with pb.load_dataset() for duckdb on Windows #245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

matt-humphrey
Copy link
Contributor

Summary

There was an error with loading temporary file for duckdb on Windows.
I changed the code to explicitly close the Ibis connection, and that seems to have resolved the problem.

Related GitHub Issues and PRs

@matt-humphrey
Copy link
Contributor Author

matt-humphrey commented Jul 17, 2025

My bad, I should have run all tests before pushing changes.

From what I gather, when pb.load_dataset() is used with duckdb, it's returning a live connection to the Ibis table. Hence a bunch of these tests failed, because I explicitly closed the connection.

The problem appears to be that Windows runs into permission errors due to maintaining that live connection beyond the function where that connection is defined. I'm not sure what an optimal solution is here - possibly converting the file into memory, and then closing the connection?

[EDIT]

Upon further investigation, part of the problem likely has to do with the test fixtures also returning live connections, for example:

@pytest.fixture
def tbl_dates_times_text_duckdb():
    file_path = pathlib.Path.cwd() / "tests" / "tbl_files" / "tbl_dates_times_text.ddb"
    with tempfile.TemporaryDirectory() as tmp:
        fpath: Path = Path(tmp) / "tbl_dates_times_text.ddb"
        shutil.copy(file_path, fpath)
        return ibis.connect(f"duckdb://{fpath!s}").table("tbl_dates_times_text")

Part of the solution might be changing these fixtures also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant