Skip to content

Frozen2 serialized representation not portable across different byte orders #668

@mhx

Description

@mhx

I'm using Frozen2 for metadata storage in a read-only filesystem. In the process of porting the code to big-endian architectures, I realized that the metadata produced on little-endian systems wasn't readable. In fact, Frozen2 cannot currently be used at all on big-endian systems: Frozen2 cannot correctly read back its own serialized representation produced on big-endian.

The root cause is that folly::Bits, which is the backbone of Frozen2, does not work correctly on big-endian. I don't think folly::Bits was designed to be portable, as the code explicitly mentions that the storage is expected to be little-endian.

Fortunately, it is relatively straightforward to fix folly::Bits by enforcing the assumption of little-endian storage using folly::Endian. There's a pull request that does just that: facebook/folly#2484. The added code is a no-op on little-endian systems, so it shouldn't introduce a performance regression.

With that change, Frozen2 works fine on big-endian systems and the serialized representation can be freely exchanged between different byte orders.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions