Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for large files (ZIP64) #30

Closed
i4rilu opened this issue Jan 6, 2022 · 6 comments · May be fixed by archiverjs/node-compress-commons#90
Closed

Support for large files (ZIP64) #30

i4rilu opened this issue Jan 6, 2022 · 6 comments · May be fixed by archiverjs/node-compress-commons#90

Comments

@i4rilu
Copy link

i4rilu commented Jan 6, 2022

AES-256 password-protected zip archive with a >4GB file inside is not correctly recognized / unpacked with the latest version of 7-Zip (21.07).
Tried to force ZIP64 with forceZip64: true with no luck (ZIP64 headers are added but 7-Zip still can't correctly recognize it).

✔️ AES-256 password-protected zip archive with a small file:

7-Zip 21.07 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26

Scanning the drive for archives:
1 file, 61183895 bytes (59 MiB)

Listing archive: *202201040400.bak.zip

--
Path = *202201040400.bak.zip
Type = zip
Physical Size = 61183895

----------
Path = *202201040400.bak
Folder = -
Size = 615973376
Packed Size = 61183691
Modified = 2022-01-04 04:03:42
Created =
Accessed =
Attributes =  -rw-r--r--
Encrypted = +
Comment =
CRC = 06D9EE07
Method = AES-256 Deflate
Characteristics = WzAES : Encrypt Descriptor
Host OS = Unix
Version = 51
Volume Index = 0
Offset = 0

❌ AES-256 password-protected zip archive with a >4GB file:

7-Zip 21.07 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26

Scanning the drive for archives:
1 file, 849018230 bytes (810 MiB)

Listing archive: *202201060617.bak.zip

--
Path = *202201060617.bak.zip
Type = zip
WARNINGS:
Headers Error
Physical Size = 849018230

----------
Path = *202201060617.bak
Folder = -
Size = 4369926656
Packed Size = 849017997
Modified = 2022-01-06 06:20:34
Created =
Accessed =
Attributes =  -rw-r--r--
Encrypted = +
Comment =
CRC = 1456C009
Method = ZipCrypto LZFSE
Characteristics = Extra_ERROR Zip64_ERROR : Encrypt Descriptor
Host OS = Unix
Version = 45
Volume Index = 0
Offset = 0
@artem-karpenko
Copy link
Owner

artem-karpenko commented Jan 10, 2022

Hi,
thank you for the report. I've been able to reproduce the issue but have not yet identified the exact problem - although I have alreayd found a couple of things that could be improved in zip64 handling. Will investigate this deeper.

@i4rilu
Copy link
Author

i4rilu commented Jan 10, 2022

I was able to make it working and be correctly recognizable by 7-Zip / WinZip by making the following modifications:

  1. ae.setExtra(_buildAesExtraField(ae));
ae.setExtra(Buffer.concat([ae.getExtra(), _buildAesExtraField(ae)]));
  1. https://github.com/archiverjs/node-compress-commons/blob/50230e5ee441cc65aef52e154471836564bc19de/lib/archivers/zip/zip-archive-output-stream.js#L277-L285
    var extraBuf = Buffer.concat([
      zipUtil.getShortBytes(constants.ZIP64_EXTRA_ID),
      zipUtil.getShortBytes(16),
      zipUtil.getEightBytes(ae.getSize()),
      zipUtil.getEightBytes(ae.getCompressedSize())
    ], 20);

    ae.setExtra(Buffer.concat([ae.getExtra(), extraBuf]));

Hope that helps to build a fix.
Thank you for your contribution on this useful library!

@robertozmc
Copy link

robertozmc commented Jan 13, 2022

We are experiencing a similar issue when trying to extract encrypted zip file over 4GB in size. When extracting process passes ~4GB it fails.
This happens only on Windows machines when using those applications:

  • WinZip
  • WinRAR
  • 7zip

The only one that worked on Windows for us was TugZip.
We are not experiencing this issue on macOS though.

Edit.
Changes proposed by @i4rilu didn't work for us. It must be something different then.

@artem-karpenko
Copy link
Owner

Hi @i4rilu! Thank you for suggestions, appreciate them! I will try them out over this weekend and possibly merge them in.

@artem-karpenko
Copy link
Owner

@i4rilu thanks again for your suggestions, they have been introduced into master now. Until archiverjs/node-compress-commons#90 is merged I'll keep a copy of _writeCentralFileHeader in this code.
Released version 1.0.11.
cc @robertozmc

@robertozmc
Copy link

@artem-karpenko your fix did the job. Thank you and @i4rilu for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants