Skip to content

Add file #5205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from
Draft

Add file #5205

wants to merge 15 commits into from

Conversation

FlaveFlav20
Copy link

I created a file-like type for PyO3. Since file is a fundamental type in Python, I believe it should be supported by this library.

Example:

use pyo3::types::pyo3file::Pyo3File;

#[pyfunction]
fn my_custom_file(mut file: Pyo3File) -> PyResult<String> {
    let mut contents = String::new();

    file.getfile().read_to_string(&mut contents)?;

    Ok(contents)
}

I created a pyo3 type because the Python file uses three parameters that are not directly available in Rust::

  • name
  • opening mode
  • encoding (we can't know the file encoding with rust file type)

I opened this pull request to check if this approach is acceptable.

Comment on lines 147 to 234
#[cfg(unix)]
fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult<Self> {
let fd: RawFd = unsafe { crate::ffi::PyObject_AsFileDescriptor(obj.as_ptr()) };
if fd < 0 {
return Err(PyErr::fetch(obj.py()));
}

let dup_fd = dup(fd).map_err(|e| PyErr::new::<PyOSError, _>(e.to_string()))?;
let file = unsafe { File::from_raw_fd(dup_fd) };

let name: String = obj
.getattr("name")?
.extract()
.unwrap_or_else(|_| "<unknown>".to_string());

let mode: String = obj
.getattr("mode")?
.extract()
.unwrap_or_else(|_| "r".to_string());

let encoding: String = obj
.getattr("encoding")?
.extract()
.unwrap_or_else(|_| "utf-8".to_string());

Ok(Pyo3File::new(
file,
name,
mode,
encoding,
))
}

#[cfg(windows)]
fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult<Self> {
let fd: i32 = unsafe { crate::ffi::PyObject_AsFileDescriptor(obj.as_ptr()) };
if fd < 0 {
return Err(PyErr::fetch(obj.py()));
}

// Convert fd to raw HANDLE
let raw_handle = unsafe { libc::get_osfhandle(fd) };
if raw_handle == -1 {
return Err(FileConversionError::new_err("Invalid handle from fd"));
}

let mut dup_handle: HANDLE = ptr::null_mut();
let success = unsafe {
DuplicateHandle(
GetCurrentProcess(),
raw_handle as HANDLE,
GetCurrentProcess(),
&mut dup_handle,
0,
1,
DUPLICATE_SAME_ACCESS,
)
};

if success == 0 {
return Err(FileConversionError::new_err("Failed to duplicate handle"));
}

let file = unsafe { File::from_raw_handle(dup_handle as RawHandle) };

let name: String = obj
.getattr("name")?
.extract()
.unwrap_or_else(|_| "<unknown>".to_string());

let mode: String = obj
.getattr("mode")?
.extract()
.unwrap_or_else(|_| "r".to_string());

let encoding: String = obj
.getattr("encoding")?
.extract()
.unwrap_or_else(|_| "utf-8".to_string());

Ok(Pyo3File::new(
file,
name,
mode,
encoding,
))
}
}
Copy link
Member

@bschoenmaeckers bschoenmaeckers Jun 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a full review yet. But you can avoid using additional dependencies on winapi/nix by using File::try_clone.

#[cfg(unix)]
let python_handle = unsafe {
    File::from_raw_fd(fd)
};

#[cfg(windows)]
let python_handle = unsafe {
    let raw_handle = libc::get_osfhandle(fd);
    if raw_handle == -1 {
        return Err(PyOSError::new_err("Cannot convert file descriptor to RawHandle"));
    }
    File::from_raw_handle(raw_handle as _)
};


let new_handle = python_handle.try_clone()?;
// Do not steal the handle from Python, as it is still used by the python object.
mem::forget(python_handle);
Ok(new_handle)

Copy link
Member

@bschoenmaeckers bschoenmaeckers Jun 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or using the lower-level BorrowedFd/BorrowedHandle:

#[cfg(unix)]
let file: File = unsafe {
    BorrowedFd::borrow_raw(fd)
        .try_clone_to_owned()?
        .into();
};

#[cfg(windows)]
let file: File = unsafe {
    let raw_handle = libc::get_osfhandle(fd);
    if raw_handle == -1 {
        return Err(PyOSError::new_err("Cannot convert file descriptor to RawHandle"));
    }
    BorrowedHandle::borrow_raw(raw_handle as _)
    .try_clone_to_owned()?
        .into()
};

Ok(file)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the winapi/nix dependencies and used File::try_clone. I also added a test.

#[cfg_attr(windows, link(name = "pythonXY"))]
extern "C" {
#[cfg_attr(PyPy, link_name = "PyPyFile_Type")]
pub static mut PyFile_Type: PyTypeObject;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this symbol exported in python 3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants