Questions

[julkhami]

Hey,

I was curious, how effective is this tool at segmenting text? Can it also tokenise words?

Can we download a pre-trained model or do we have to train it ourselves? If so, why is that, just out of curiosity?

Is this one of the best such tools out there at the moment or is there any standard tool or machine learning library at the moment which offers a method like this?

Thanks very much.