Speed question

[vince62s]

Hello,

I have not tested the package yet but I am interested in speed comparison vs CLD3 or langid (python and C versions)

did you benchmark from that perspective ?

Thanks.

[pemistahl]

Hi Vincent, thanks for your question and your interest in my library.

Compared to CLD3 or langid, Lingua is slower. Lingua uses much larger language models and is currently implemented in pure Python only. Some time in the future, I will try to implement key parts in Cython to speed it up. If runtime performance in terms of speed is crucial for your use case, you can try my Go or Rust implementations of the same library. They perform significantly faster than the Python one.

[vince62s]

thanks for your feedback. Indeed speed is an issue for my use case.
I found this paper: https://arxiv.org/pdf/1910.06748.pdf
unrelated to Lingua project but I found some data on peed and accuracy. Just surprised that CLD3 is bechmarked lower than CLD2 (in comparison to your benchmarks)

[pemistahl]

Thanks for the paper, looks interesting. :) I'll read it once I find the time.