Creating vrt files for cqpweb

[tianshuo]

Feature description

We want to find process text into an annotated corpus and search on it using POS+regex. For example, if we want to search for "look|looking|looked for", we can use
"{look/V} for"
We already have been using spacy's pattern matching function but it is a bit slow(because we don't index anything) and we found out cqpweb is a great tool for both indexing, searching (it has a special syntax that works for us) and has a web UI. So the next step is we need to export everything into the xml-based .vrt files used in Cqpweb. See the instructions here: http://cwb.sourceforge.net/files/CWB_Encoding_Tutorial.pdf

An example format:

<?xml version="1.0" encoding="ISO-8859-1" standalone="yes" ?>
<!-- A Thrilling Experience -->
<story num="4" title="A Thrilling Experience">
<p>
<s>
Tick NN tick
. SENT .
</s>
<s>
A DT a
clock NN clock
. SENT .
</s>
<s>
Tick VB tick
, , ,
tick VB tick
. SENT .
</s>
</p>
...
</story>

Could the feature be a custom component or spaCy plugin?

Yes! This could be a spaCy plugin, where we can export our corpus as the .vrt file. Or we could even create a tool for importing managing, browsing and searching text in our corpus.

[adrianeboyd]

I'd recommend looking at how textacy implements doc_to_conll(), since this should be a pretty similar task:

https://github.com/chartbeat-labs/textacy/blob/5156268e6cfcf45fc55a20504a02019697f42a2b/textacy/export.py#L81-L132

[RaulKite]

Did you use this? I'm very interested going inside this task and searching in Google have found this thread.

I'll appreciate any advice or info on this since this thread has 3 years old before starting to write my own code

Thanks

[tianshuo]

Write it manually, should take only a few hours of development.