Skip to content

Replace jsunzip with modern libraries (fflate + pako) #506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 2, 2025

Conversation

mattgodbolt
Copy link
Owner

@mattgodbolt mattgodbolt commented Jul 2, 2025

Summary

  • Replace custom jsunzip library (628 lines) with modern maintained libraries
  • Use fflate for ZIP operations (synchronous unzipSync API)
  • Use pako for gzip operations (concatenated stream support)
  • Remove legacy src/lib/jsunzip.js file completely
  • Maintain all existing APIs: unzipDiscImage(), unzipRomImage(), ungzip()

Technical Decisions

  • ZIP: fflate chosen over JSZip for its synchronous API (no promises/async required)
  • Gzip: pako chosen over fflate for concatenated gzip stream support (standard format)
  • No fallbacks: Each library handles its specific use case reliably
  • Incremental migration: Committed working pieces separately for safe rollback

Testing

  • ✅ All 241 unit tests pass
  • ✅ All integration tests pass
  • ✅ All CPU compatibility tests pass
  • ✅ Added comprehensive ZIP functionality tests
  • ✅ Existing gzip tests continue to work

Files Changed

  • src/utils.js: Updated compression/decompression functions
  • package.json: Added fflate and pako dependencies
  • src/lib/jsunzip.js: Removed (628 lines deleted)
  • src/lib/README: Updated to remove jsunzip reference
  • tests/unit/test-zip.js: New comprehensive ZIP tests
  • New test ZIP files in tests/unit/zip/ directory

🤖 Generated with Claude Code

mattgodbolt and others added 4 commits July 1, 2025 18:35
- Remove custom TINF gzip implementation in favor of maintained library
- Part of broader effort to replace jsunzip with standard libraries
- Maintains existing ungzip() API and error handling
- All existing gzip tests continue to pass

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Remove custom JSUnzip implementation in favor of maintained library
- Use fflate's unzipSync for synchronous ZIP handling
- Add comprehensive ZIP tests with sample files
- Maintains existing unzipDiscImage() and unzipRomImage() APIs
- All existing functionality continues to work
- Fix lint-staged to only run prettier on supported file types

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Delete custom jsunzip implementation (628 lines)
- Update lib/README to remove jsunzip reference
- All tests continue to pass with new fflate/pako implementation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@mattgodbolt
Copy link
Owner Author

Should lay a tiny bit of groundwork for #505 (though not directly in this form)

@mattgodbolt mattgodbolt requested a review from Copilot July 2, 2025 00:46
@mattgodbolt mattgodbolt marked this pull request as ready for review July 2, 2025 00:46
Copilot

This comment was marked as outdated.

- Optimize Uint8Array usage to avoid unnecessary buffer copying
- Update test-zip.js to use vitest expect API for consistency
- Fix path handling to use path.join for cross-platform compatibility

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@mattgodbolt mattgodbolt requested a review from Copilot July 2, 2025 02:25
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR replaces the custom jsunzip implementation with maintained libraries and updates related tests and docs.

  • Swap out custom ZIP/GZIP code for fflate.unzipSync and pako.ungzip in src/utils.js
  • Remove the legacy src/lib/jsunzip.js file and references
  • Add comprehensive ZIP tests covering SSD/ROM, mixed archives, and error cases

Reviewed Changes

Copilot reviewed 5 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/utils.js Replaced jsunzip logic with fflate and pako calls
src/lib/jsunzip.js Deleted legacy JSUnzip library
tests/unit/test-zip.js Added new ZIP unit tests for various archive scenarios
src/lib/README Removed jsunzip reference
package.json Added fflate and pako dependencies, updated formatting rules
Comments suppressed due to low confidence (1)

tests/unit/test-zip.js:49

  • [nitpick] The comment says this ZIP is incompatible but the test loads test-ssd.zip, which actually contains a valid SSD file; please update the comment to match the test data.
        // Create a simple ZIP with incompatible file

@@ -1030,35 +994,38 @@ const knownRomExtensions = {
};

function unzipImage(data, knownExtensions) {
const unzip = new jsunzip.JSUnzip();
console.log("Attempting to unzip");
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider removing or replacing this debug console.log with a proper logger or debug-level logging to avoid polluting production output.

Copilot uses AI. Check for mistakes.

if (!match || !knownExtensions[match[1].toLowerCase()]) {
console.log("Skipping file", f);
console.log("Skipping file", filename);
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] This debug console.log could be removed or switched to a debug logger to keep production logs clean.

Copilot uses AI. Check for mistakes.

continue;
}
if (uncompressed) {
console.log("Ignoring", f, "as already found a file");
console.log("Ignoring", filename, "as already found a file");
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider removing or demoting this console log to a debug channel to prevent excessive log output in production.

Copilot uses AI. Check for mistakes.

if (!uncompressed.status) {
throw new Error("Failed to uncompress file '" + loadedFile + "' - " + uncompressed.error);
}

console.log("Unzipped '" + loadedFile + "'");
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] This final console.log is useful for debugging but might be noisy in a production environment; consider using a configurable logger.

Copilot uses AI. Check for mistakes.

@mattgodbolt mattgodbolt merged commit 0a5cada into main Jul 2, 2025
3 checks passed
@mattgodbolt mattgodbolt deleted the claude/replace-jsunzip-with-fflate branch July 2, 2025 02:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant