Textcat or NER ?

[yellowy]

As my first software project ever I'm trying to build a harmonizing tool for our DB and I thought spaCy might help me a great deal with that. I get databases from customers regularly and those csv files sometimes arrive without a GTIN (with which i would harmonize the data so we can continue to work with the data)
So I thought i would let a spacy model learn the names of the products and I was wondering how to do this in a smart way.

I thought I should either train a SpaCy model on a combined GTIN and product name and then let this run on foreign databases and whenever he identifies a name he can find the corresponding GTIN for it? or would a different approach be better?

And is NER or Textcat better suited for this kind of task?
The Data I receive is like this:
"id; product-name; packaging" and so on

I'm sorry if this is too low level of a question but I'm very thankful for any nudge in the right direction.

[bdura]

Hello @yellowy, thanks for using spaCy! I'd explore using rule-based pattern matching in your case. You can create patterns quickly with spaCy's PhraseMatcher.

[yellowy]

Thanks a lot for your response!

But are you sure? I feel like it's gonna be very impractical to do patterns for data that's very unstructured like this :

GTIN
0123456789123 J. Daniels 0,7 L Bottle
0123456789123 Jack D. 0.7LBott(l)e
0123456789123 J.D. 0,7Litre(Bottle)
4567891234561 J. Daniel 1.0 L Bottle
4567891234561 Jack Daniels Tennese 1LBottle
J. Daniels 0,7L Bottle
Daniels, Jack Bottle(1.0L)

and so on.

I fear that there is not really a general pattern going on, and the patterns I can find manually are probably not gonna be very effective for big amounts of data or am I wrong?

What I thought I could do is make a blank model, train a textcat where i use the GTIN as a Key and the Product Names as the value. Then use that trained model(with the saved GTIN's as keys) to find out the GTIN of new data where the GTIN is missing due to various reasons but where the naming is similar enough that i hope the trained model can pick up different patterns that fit to various products, thus finding the corresponding GTIN with a certain amount of confidence

Thanks a lot for taking your time!

[bdura]

It really depends on the shape of your data, the number of different GTIN and the variability of the patterns. spaCy pipelines are token-based, and as such there isn't much information to work with for an ML pipeline if most product names span a single token.

Seeing your data, it seems like you might benefit from using a textcat component. Your dataset should let you generate a training corpus easily (see here for an example) so you can try this out.