Skip to content

Optimize serialization of small integers #5482

@fingolfin

Description

@fingolfin

Although this provides a speed up we would need to introduce a limit on Ints to be serialized as number since they could be turned into floats depending on the implementation of the JSON reader. Here we serialize Ints as JSON Numbers when the integer is between 1 - 2^53 and 2^53 - 1 and strings otherwise. However on load we need to check type to know whether we need to parse or not, and this is a big drawback.

I think it would be worth a try, in a separate branch, to try exactly that: whenever an Int is between 1 - 2^53 and 2^53 - 1, we write it as an Int/Float, otherwise as a string. Then, for a semi-randomly selected set of test cases, we can compare the file sizes and timings for write and read.

FYI: @benlorenz

Originally posted by @micjoswig in #4558

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions