Minimum hardare

[dade2906]

are there any minimum hardware requirements for running a classification algorithm using spacy in particular for 200.000 email? If so, is there any official documentation about it?

[svlandeg]

Hi!

It really depends on the type of classification algorithm you want to run, and whether you'd use a tok2vec- or a transformer-based pipeline. You'd need at least a GPU for the latter.

Also consider that you might want to consider different hardware for training (more memory) than for just "running" the algorithm in prediction mode, which is less demanding. And you can parallellize the 200.000 emails over different machines / cores even outside of the spaCy prediction loop.

In general, if you use a tok2vec-based textcat algorithm, you won't need any fancy hardware and you can just run it on a CPU. If the documents are long however, you might need more memory, or consider splitting up the docs in segments.