-
Notifications
You must be signed in to change notification settings - Fork 162
Description
Although this provides a speed up we would need to introduce a limit on Ints to be serialized as number since they could be turned into floats depending on the implementation of the JSON reader. Here we serialize Ints as JSON Numbers when the integer is between 1 - 2^53 and 2^53 - 1 and strings otherwise. However on load we need to check type to know whether we need to parse or not, and this is a big drawback.
I think it would be worth a try, in a separate branch, to try exactly that: whenever an Int is between 1 - 2^53 and 2^53 - 1, we write it as an Int/Float, otherwise as a string. Then, for a semi-randomly selected set of test cases, we can compare the file sizes and timings for write and read.
FYI: @benlorenz
Originally posted by @micjoswig in #4558