user_data or UnderScore in spacy.kb.Candidate

[tamuhey]

Feature description

If spacy.kb.Candidate can contain more information, it is very useful, e.g. it can represent knowledge graph node.
Like Doc, I think it can be achiedved if Candidate have user_data or UnderScore.

[narayanacharya6]

It's been close to 2 years now since this issue was posted. I am guessing this is not an immediate priority but I was hoping if someone from the core team could give a brief roadmap of how this can be achieved. I am guessing apart from the extra information required on the Candidate class, it actually requires changes in the KnowledgeBase class to actually hold these extra fields (which also need to indexed appropriately for fast lookups?). I am not really experienced with Cython which is used generously in the KnowledgeBase source code, so any pointers would be really helpful :)

[kinghuang]

I'm wondering this myself. Looking at the code, I imagine you'd have to override KnowledgeBase.add_entity/set_entities to store the extra information, then KnowledgeBase.get_alias_candidates and the Candidate class to return the extra information. Plus modify from_disk/to_disk for serialization.

[polm]

For anyone interested in this feature, could you provide some more examples of how you'd use it?

My initial take on this is that it's a bad idea for a few reasons. The KB is designed as a minimal, efficient lookup that cooperates with the NEL model. It's not really suitable as a source of truth or for long term data management, and is presumably based on external data. Since you're already managing this external data somehow, it makes sense to me to incorporate look things up in it by ID in your application flow.

Another thing is, if you rely on the KB giving you all your data, 1. there's more data in the KB so you run into scaling problems more quickly 2. when you run into scaling problems, the easiest way to fix them will be to implement an ID lookup in an external store (to reduce KB size), which puts us back at not having extra data in the KB.

The main case I can see this being frustrating is for small KBs where it's easier to have everything in one place, but I still think using an ID to look things up in a dict isn't that bad. If you feel differently or have a use case where what I described above doesn't hold I'd be happy to hear about it though.