Pyodide NODEFS filesystem writes fail with invalid length error

[mlilback]

Bug Report: Pyodide NODEFS Filesystem Writes Fail in Deno

Summary

Pyodide's NODEFS filesystem mount succeeds in Deno, but file write operations fail with OSError: raw write() returned invalid length. The same code works correctly in Node.js, indicating a Deno-specific incompatibility with Emscripten's NODEFS implementation.

Status: Confirmed still broken in Deno 2.5.4 (latest stable release as of October 2025)

Environment

• Deno version: 2.5.4 (latest stable as of 2025-10-10)
• Platform: macOS 15.6.1 (darwin aarch64)
• Pyodide version: 0.27.5
• V8 version: 14.0.365.5-rusty
• TypeScript version: 5.9.2

Expected Behavior

When mounting a host directory to Pyodide's virtual filesystem using NODEFS and writing files through Python, the files should be written to the host filesystem successfully (as they are in Node.js).

Actual Behavior

The NODEFS mount succeeds without error, but attempting to write files through Python results in:

OSError: raw write() returned invalid length 3285164 (should have been between 0 and 13)

The error shows raw write() returning an impossibly large value (3,285,164 bytes) when trying to write a small string (13 bytes: "Hello, World!").

Minimal Reproduction

Save the following as nodefs-bug-reproduction.ts:

/**
 * Minimal reproduction of NODEFS write failure in Deno
 *
 * Expected: File writes through Pyodide's NODEFS-mounted filesystem should succeed
 * Actual: OSError: raw write() returned invalid length (much larger than expected)
 *
 * This works correctly in Node.js but fails in Deno.
 */

import { loadPyodide } from "npm:pyodide@0.27.5";

console.log("Deno version:", Deno.version);
console.log("Platform:", Deno.build.os, Deno.build.arch);
console.log();

// Create temporary directory on host filesystem
const tempDir = await Deno.makeTempDir();
console.log("Host directory:", tempDir);

// Load Pyodide
console.log("Loading Pyodide...");
const pyodide = await loadPyodide({ fullStdLib: false });
console.log("Pyodide version:", pyodide.version);
console.log();

// Create virtual directory in Pyodide
pyodide.FS.mkdir('/test');

// Attempt to mount NODEFS
console.log("Mounting NODEFS...");
try {
    pyodide.FS.mount(
        pyodide.FS.filesystems.NODEFS,
        { root: tempDir },
        '/test'
    );
    console.log("✓ NODEFS mount succeeded");
} catch (error) {
    console.error("✗ NODEFS mount failed:", error);
    await Deno.remove(tempDir, { recursive: true });
    Deno.exit(1);
}

// Attempt to write a file through Python
console.log("\nWriting file through Python...");
try {
    pyodide.runPython(`
with open('/test/hello.txt', 'w') as f:
    f.write('Hello, World!')
print('✓ File written successfully')
    `);

    // Verify file on host
    const content = await Deno.readTextFile(`${tempDir}/hello.txt`);
    console.log("✓ File verified on host:", content);
    console.log("\n✓ SUCCESS: NODEFS is working!");
} catch (error) {
    console.error("\n✗ FAILURE: File write failed");
    console.error("Error type:", error.constructor.name);
    console.error("Error message:", error.message);
    console.error("\nFull error:", error);
}

// Cleanup
await Deno.remove(tempDir, { recursive: true });

Run with:

deno run --allow-all nodefs-bug-reproduction.ts

Output

Deno version: { deno: "2.5.4", v8: "14.0.365.5-rusty", typescript: "5.9.2" }
Platform: darwin aarch64

Host directory: /var/folders/c4/9615f6ks2jb48xrjyt7scrd40000gn/T/8bbca84df330364
Loading Pyodide...
Pyodide version: 0.27.5

Mounting NODEFS...
✓ NODEFS mount succeeded

Writing file through Python...

✗ FAILURE: File write failed
Error type: PythonError
Error message: OSError: raw write() returned invalid length 3285164 (should have been between 0 and 13)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/lib/python312.zip/_pyodide/_base.py", line 523, in eval_code
    .run(globals, locals)
     ^^^^^^^^^^^^^^^^^^^^
  File "/lib/python312.zip/_pyodide/_base.py", line 357, in run
    coroutine = eval(self.code, globals, locals)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<exec>", line 2, in <module>
OSError: raw write() returned invalid length 3285164 (should have been between 0 and 13)

Analysis

The issue appears to be in how Deno implements the fs.write() or fs.writeSync() functions that Emscripten's NODEFS relies on. The returned byte count is nonsensical (3,285,164 instead of 13), suggesting:

1. A return value mismatch between what NODEFS expects and what Deno provides
2. Possible incorrect type coercion or promise handling
3. Memory address being returned instead of byte count

This is particularly problematic because:

• NODEFS is the only Pyodide filesystem that bridges virtual FS to host FS in non-browser environments
• Without it, files written in Pyodide are invisible to the host process (requiring manual extraction)
• This significantly impacts any Deno application using Pyodide for Python code execution with file I/O

Related Issues

This issue has been reported before but appears unresolved:

• Deno Issue Can't write files in Emscripten's NODEFS when using Pyodide #23181: "Can't write files in Emscripten's NODEFS when using Pyodide"

  • Can't write files in Emscripten's NODEFS when using Pyodide #23181
  • Originally reported April 2024 with Deno 1.43.1
  • Closed as "Upstream reported fixed" but issue persists in Deno 2.5.4 (latest stable)

• Emscripten Issue Crash when trying to use the addEventListener on GPUDevice #21673: "NODEFS not compatible with Deno when used through Pyodide"

  • NODEFS not compatible with Deno when used through Pyodide emscripten-core/emscripten#21673
  • Notes that Emscripten does not officially support Deno

Workaround

We've implemented a workaround by:

1. Using Pyodide's default MEMFS (in-memory filesystem)
2. Extracting files from Pyodide's virtual FS to host FS after Python execution completes
3. Using pyodide.FS.readFile() and Deno.writeFile() for manual synchronization

This works but adds complexity and requires careful file lifecycle management.

Request

Can the Deno team investigate the incompatibility between Deno's filesystem implementation and Emscripten's NODEFS expectations? Specifically:

1. What does Deno's fs.write()/fs.writeSync() return that differs from Node.js?
2. Can Deno's implementation be adjusted to be compatible with Emscripten's expectations?
3. If not, should Deno document this limitation clearly?

This would enable proper Pyodide filesystem integration in Deno without manual workarounds.

Additional Testing

Willing to provide additional test cases or debugging information if helpful. Can also test on Linux if needed to confirm this is not macOS-specific.

---

Priority: Medium-High - Affects any Deno application using Pyodide with file I/O

Impact: Workarounds exist but require significant additional code and complexity

[mlilback]

I've filed corresponding issues with the other affected projects to ensure all teams are aware:

• Pyodide Deno.setRaw() throws OS error 6 on Windows #5943: NODEFS filesystem incompatible with Deno runtime

  • NODEFS filesystem incompatible with Deno runtime (works in Node.js) pyodide/pyodide#5943
  • Asking if Pyodide can provide a Deno-compatible filesystem or better document the limitation

• Emscripten Memory leak with ky library #25546: NODEFS fails in Deno runtime with "raw write() returned invalid length" error

  • NODEFS fails in Deno runtime with "raw write() returned invalid length" error emscripten-core/emscripten#25546
  • Requesting investigation of what Deno's fs.write() returns that breaks NODEFS assumptions

[mlilback]

Root Cause Identified

After debugging with the Emscripten and Pyodide teams, I've identified the root cause of the NODEFS failure.

The Issue

Pyodide's NODEFS implementation uses the modern Node.js fs.writeSync API with an options object:

fs.writeSync(fd, buffer, {position: position})

However, Deno's node:fs compatibility layer doesn't support this syntax. It only supports the traditional 5-parameter API:

fs.writeSync(fd, buffer, offset, length, position)

Direct Test

import * as fs from "node:fs";
const fd = fs.openSync("/tmp/test.txt", "w");
const buffer = Buffer.from("test");

// Traditional API - Works ✓
fs.writeSync(fd, buffer, 0, 4, 0);  // Returns: 4

// Modern API - Fails ✗
fs.writeSync(fd, buffer, {position: 0});
// Error: The "offset" argument must be of type number.
//        Received an instance of Object

Two Viable Fixes

Option 1: Pyodide Uses Traditional Signature

• Pyodide modifies NODEFS to use: fs.writeSync(fd, buffer, 0, length, position)
• Pros: Works with all Node.js-compatible runtimes (current Deno, older versions, other platforms)
• Pros: Matches Emscripten's original NODEFS implementation
• Cons: Pyodide needs to change their code
• Status: Reported to pyodide/pyodide#5943

Option 2: Deno Supports Options Object Syntax

• Deno enhances node:fs.writeSync to support: fs.writeSync(fd, buffer, {position: pos})
• Pros: Better Node.js compatibility (matches full Node.js API surface)
• Pros: Other libraries using the options syntax would also benefit
• Cons: Doesn't help users on older Deno versions
• Cons: Pyodide would need runtime detection for compatibility

Node.js API Documentation

According to Node.js fs.writeSync documentation, both signatures are valid:

1. fs.writeSync(fd, buffer[, offset[, length[, position]]])
2. fs.writeSync(fd, buffer[, options])

Where options can be: { offset?, length?, position? }

Deno currently only implements signature #1.

Impact

This affects any library using WASM filesystem operations in Deno, particularly:

• Pyodide (Python in browser/Deno)
• Other Emscripten-based tools that use NODEFS
• Any code expecting full Node.js fs API compatibility

Both Fixes Could Happen

Ideally both fixes would be implemented:

1. Pyodide fix (short-term): Use traditional signature for universal compatibility
2. Deno enhancement (long-term): Complete Node.js fs API parity

This way, Pyodide would work immediately in all Deno versions, and future Deno versions would have better Node.js compatibility for other libraries.

My Workaround

I've implemented a workaround using MEMFS + manual file extraction, so this isn't blocking my project. But fixing this would benefit the broader Deno + WASM ecosystem.

Related Issues

• emscripten-core/emscripten#25546 - Debugging investigation
• pyodide/pyodide#5943 - Pyodide-side fix proposal

Environment

• Deno: 2.1.4 (Darwin 24.6.0, aarch64)
• Pyodide: 0.27.7
• Node.js fs.writeSync: Both signatures supported
• Deno node:fs.writeSync: Only traditional signature supported

Thanks for looking into this!

[bartlomieju]

@Tango992 PTAL