FP16 input support

[skill-diver]

Hi Optimum guys,

Do you know how to enable fp16 input when use your quantized model? I find qconv2d don't support fp16 input.

[dacorvo]

You should load the model with dtype=torch.float16, then quantize it. Do you have a specific code snippet that reproduces the error you are facing ?

[skill-diver]

Hi, thanks for answering. I get this error when I use mix precision to Inference quantized model.

[LukeLIN-web]

dtype=torch.float16

It seems that your weight is still fp32, Do you have a specific code snippet that reproduces the error you are facing ?