Model data type

[ThomAub]

Hello,

Will it be possible to configure model data type (same a tensor dtype) ?

Let's say it do some quantisation on my model I would like to be able to use redis AI and do inference on tensor in float16, int8.

I think most backend implement this mechanism but I don't know if it's something that would interest redisAI ?

[gkorland]

Sure, RedisAI support different datatypes see TENSOR.SET command: https://oss.redislabs.com/redisai/commands/#aitensorset

[ThomAub]

Hello,

thank you for your answer and sorry I wasn’t very clear in my question.

In AI.TENSORSET it’s possible to add the datatype in the tensor META.
For AI.MOSELSET or AI.MOSELSTORE it’s not. Would it be something of interest to add a dtype in the model META ? This would represent the model weights datatype.

It’s not really an issue but just an idea for discussion.
I’m currently overcoming this with the tag field.

thanks again

[gkorland]

We didn't think about since the models are usually holding this metadata in the graph definition