Applying INT8 quantization, just like in Dorado

[wtrinh8]

Hello all,

I was curious how actively Bonito is still being used, as I read that Dorado nowadays converted a majority of its neural network code to INT8.

I was interested in experimenting with the current Dorado model, but I could not find a way to create an ONNX model export of the C++ neural network model.

Therefore, is there still a possibility that optimization features from Dorado will be applied to Bonito as well?

Kind regards

[iiSeymour]

Hi @wtrinh8

All models that dorado provides come from (and are available in) bonito so it's still active but note from the README:

For anything other than basecaller training or method development please use dorado.

So, no, most optimisation features from Dorado will not be applied to Bonito.

If you want to export a model to ONNX you are better off doing it from bonito.

HTH

[Dragonsky8]

Hi, I was looking into the Koi lstm.py file and saw you could apply quantisation there via a flag. However, the [email protected] does not use the LSTM class from Koi. Is it possible to use the LSTM from Koi, instead of the LSTM from Torch?