Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(handler): add partclone handler #1155

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

rxpha3l
Copy link
Contributor

@rxpha3l rxpha3l commented Mar 26, 2025

Partclone is a program similar to the well-known backup utility "Partition Image" a.k.a partimage. Partclone provides utilities to save and restore used blocks on a partition and is designed for higher compatibility of the file system by using existing libraries, e.g. e2fslibs is used to read and write the ext2 partition.
Partclone now supports ext2, ext3, ext4, hfs+, reiserfs, reiser4, btrfs, vmfs3, vmfs5, xfs, jfs, ufs, ntfs, fat(12/16/32), exfat, f2fs, nilfs.
Partclone has two version. This handler supports the second one and automatically checks the endian value.

Create partclone image:
partclone.[file_system] -F -c -s [source_file] -o [output.img] -L /dev/stdout[default=/var/log/partclone.log]

Restore partclone image:
partclone.restore -W -r -s [source_fiile.img] -o [restored.img] -L /dev/stdout[default=/var/log/partclone.log]

Notes

  • It is worth to mention that a logfile -L should be used since it will lead in the most cases to errors
  • Partclone has no standardized file extension but the official website uses .img
  • The official documentation got the endian values wrong. Swapping them will solve this issue

The header has the following structure :

16 bytes : Magic
14 bytes : Partclone version
4 bytes : Image version in str
2 bytes : Endian
16 bytes : File system type
8 bytes : File system size
8 bytes : File system total block count
8 bytes : Used blocks based on superblock
8 bytes : Used blocks based on bitmap
4 bytes : Block size
4 bytes : Feature size
2 bytes : Image version in bin
2 bytes : Number of bits for CPU
2 bytes : Checksum mode
2 bytes : Checksum size
4 bytes : Blocks per checksum
1 byte : Reseed checksum
1 byte : Bitmap mode
4 bytes : CRC32 of previous 106 bytes

Unblob parses the header to determine the start and end offset by adding:

  • Header
  • Bitmap size + Checksum
  • Data size
  • Checksum per N bytes. This is calculated by multiplying Block size*Blocks per checksum. Each N-Bytes a 4 byte checksum will occur (min: 1)
    Finally the CRC32 value in the header does not the final XOR with 0xFFFFFFFF. This can lead to deviation if checked manually.

[Sources]
https://github.com/Thomas-Tsai/partclone
https://partclone.org/
https://wiki.osdev.org/Ext2
https://piazza.com/class_profile/get_resource/il71xfllx3l16f/inz4wsb2m0w2oz
https://www.science.smith.edu/~nhowe/262/oldlabs/ext2.html
https://www.science.smith.edu/~nhowe/262/oldlabs/kernel/ext2_fs.h

@qkaiser qkaiser self-requested a review March 26, 2025 14:51
@qkaiser qkaiser added documentation Improvements or additions to documentation enhancement New feature or request format:archive dependencies Pull requests that update a dependency file labels Mar 26, 2025
@qkaiser qkaiser added this to the Internship 2025 milestone Mar 26, 2025
@qkaiser qkaiser linked an issue Mar 26, 2025 that may be closed by this pull request
Copy link
Contributor

@qkaiser qkaiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job :)

rxpha3l and others added 3 commits March 28, 2025 09:15
Instead of adding patches on top of unlob package available on nixpkgs,
we keep our own package.nix version here so that it's easy to add new
third party dependencies that new handlers may require.

It also makes it easier to upstream it since we just have to submit the
package.nix file to nixpkgs.
Partclone is a utility used for backing up and restoring partitions.
Many cloning tools (such as Clonezilla) rely on it to create block-level
images that include filesystem metadata.

Right now only partclone version 2 is supported. end offset is computed
from data available in partclone's super block.

Extraction is performed by `partclone.restore`, which is part of
`partclone` package on Debian based systems and Nix.
@qkaiser
Copy link
Contributor

qkaiser commented Mar 28, 2025

partclone is not available on OSX in nixpkg - https://github.com/onekey-sec/unblob/actions/runs/14124724723/job/39571274718?pr=1155

Would need to provide a patch for https://github.com/NixOS/nixpkgs/blob/e6057380b59bdf9a3693ca3ef45d67e6a95a0d66/pkgs/by-name/pa/partclone/package.nix.

Will look into it. It will take some time since I need to spin up some VMs.

@qkaiser
Copy link
Contributor

qkaiser commented Apr 11, 2025

I looked into partclone, it can't be built on OSX. It depends on libraries that are only available on Linux (e.g. e2fsprogs or nilfs) and the code needs header files that can only be found on Linux (e.g. malloc.h, features.h).

We need the ability to tag handlers as unsupported on OSX so their test is not run when testing on that platform. Ideas welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation enhancement New feature or request format:archive
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Support for Partclone Images
2 participants