Skip to content

Conversation

stalkopat
Copy link
Contributor

@stalkopat stalkopat commented May 20, 2025

Motivation

Ints are currently always serialized as strings, even for binary formats. This causes quite a bit of overhead depending on the downstream usage. (For example persisting to disk)
I256::MAX with the existing serialization format uses ~85 bytes with bincode, v.s. 40 bytes with this change

Solution

Use is_human_readable to serialize Ints as compact binary blobs instead.

PR Checklist

  • Added Tests
  • Added Documentation
  • Breaking changes

Considerations

This change isn't backwards compatible with already serialized data that has been f.x. persisted to disk with a non human readable format. Keeping the deserialize_any in the deserialize should technically work, but apparently some of the self describing formats just treat strings and bytes interchangeably, which means those will face compatibility issues anyway.

Making this change has the additional benefit of now allowing f.x. bincode to work with ints (because it gets rid of the deserialize_any in the non readable path).

Json and related readable formats (which I assume is the majority of downstream usage) shouldn't be impacted.

@stalkopat
Copy link
Contributor Author

stalkopat commented May 20, 2025

On a second thought, up until ~2^103 the string representation is actually smaller than the fixed 32 byte representation, this might do more harm than good for the vast majority of use cases. Definitely think deserialize_any should go for non human readable formats though. Just serializing those as string doesn't quite feel right either.

@stalkopat stalkopat closed this May 20, 2025
@github-project-automation github-project-automation bot moved this to Done in Alloy May 20, 2025
@stalkopat stalkopat reopened this May 21, 2025
@github-project-automation github-project-automation bot moved this from Done to In Progress in Alloy May 21, 2025
@DaniPopes DaniPopes changed the title Serialize Signed Ints with a compact binary representation feat(primitives): serialize Signed with a compact binary representation Jul 1, 2025
@DaniPopes
Copy link
Member

DaniPopes commented Jul 1, 2025

This logic is already implemented in Uint, so we can just delegate to it in the non-human-readable case. I've implemented this in a6613b2.

@github-project-automation github-project-automation bot moved this from In Progress to Reviewed in Alloy Jul 1, 2025
@DaniPopes DaniPopes enabled auto-merge (squash) July 1, 2025 20:55
@DaniPopes DaniPopes disabled auto-merge July 1, 2025 21:05
@DaniPopes DaniPopes enabled auto-merge (squash) July 1, 2025 21:25
@DaniPopes DaniPopes merged commit 0af537d into alloy-rs:main Jul 1, 2025
31 checks passed
@github-project-automation github-project-automation bot moved this from Reviewed to Done in Alloy Jul 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants