Skip to content

fix: Permission Error trying to use pb.load_data_set() with duckdb on Windows #244

@matt-humphrey

Description

@matt-humphrey

Description

When running pb.load_dataset(dataset="small_table", tbl_type="duckdb") on my Windows machine, I get the following error:

File "C:\Users\00113294\Projects\pointblank\pointblank\validate.py", line 562, in load_dataset
    with tempfile.TemporaryDirectory() as tmp, ZipFile(data_path, "r") as z:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 946, in __exit__
    self.cleanup()
    ~~~~~~~~~~~~^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 950, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 930, in _rmtree
    _shutil.rmtree(name, onexc=onexc)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\shutil.py", line 790, in rmtree
    return _rmtree_unsafe(path, onexc)
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\shutil.py", line 629, in _rmtree_unsafe
    onexc(os.unlink, fullname, err)
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 905, in onexc
    _os.unlink(path)
    ~~~~~~~~~~^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\00113294\\AppData\\Local\\Temp\\tmps3x_936b\\small_table.ddb'

According to Claude:

This is a Windows-specific issue with DuckDB file locking and temporary file cleanup. The error occurs because:

  1. File Handle Not Released: When pb.load_dataset() creates a DuckDB database file in a temporary directory, the database connection isn't being properly closed before Python tries to clean up the temporary directory.
  2. Windows File Locking: Windows is more strict about file locking than Unix systems - it won't allow deletion of files that have open handles, even if the process that opened them is trying to delete them.

Development environment

  • Operating System: Windows
  • Pointblank Version: 0.11.2

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions