Returned maps have wrong key value types.

[Audrius-GR]

Hi,

Thanks for the great library, but I've got a question/issue.

Let's say I am storing a Map<Int, Int> in a bin.
When packing, that nicely realizes that the ints are small values, and stores them as shorts server side.
Great.

Now when I read the map back out of the bin, due to this:

aerospike-client-java/client/src/com/aerospike/client/util/Unpacker.java

Lines 297 to 300 in d65f8f8

	case 0xcd: { // unsigned 16 bit integer
	int val = Buffer.bytesToShort(buffer, offset);
	offset += 2;
	return getLong(val);

I always end up reading a Map<Long, Long>.

Now what is worse, if I do something like this:

mapWritten.keys().forEach { key ->
     doSomething(mapReadBack[key])
}

I always end up getting nulls, because the int key I just wrote, does not match anything in the returned map as the map now has longs and any keep lookup fails equality due to type mismatch.

Obviously this should be a type error but JVM erasure hides it, and developers end up sinking quite some time into debugging this 😞

What's worse is that this might lead to runtime errors in applications, that expect to do integer arithmetic etc with the values they got back 😞

I assume this is not intentional?
Or if it is, where is this documented, because it's definitely a "gotcha" that caught me out.

Best wishes.

[Audrius-GR]

I also suspect it would be incredibly hard to fix this without breaking everyone who is already working around this...

[BrianNichols]

It's intentional. CDT lists/maps are stored in space efficient MessagePack format, but this format only specifies the storage type and not the original unpacked type. It was decided to not store the original unpacked type in an extra byte with every value to save space. The consequence of that decision is all integers are converted to 64 bits when read from MessagePack encoded data.

A similar approach is used for integer bin values where all integers are stored/retrieved as 64 bit integers.

The code docs do mention that MessagePack is the encoding format, but I agree that the consequences of that decision are not explained.

[AudriusButkevicius]

Replying from a different account.

I guess even if the client tried to use the most optimal type for the value at hand (for example if it's ushort server side, int is more than enough client side), you'd have the problem going the other way, where a written long comes out as an integer.

I am trying to think of a way to prevent developers from tripping on this in the future, but nothing comes to my head.

Perhaps the collections returned by the client library could be smarter, and things like Map.containsKey could check the type of the value provided, if it's a fixed point value, cast it to long before calling the underlying collection.

Obviously this is even more magical, as I suspect it will work in 90% of the cases, and then when it doesn't, the debugging session will be even longer.