Implement Twitter document source

[johann-petrak]

This has been partly done for old-format files. Need to figure out how this changes for new-style result JSON.

See also https://github.com/twitterdev/Twitter-API-v2-sample-code

[greenwoodma]

If you are talking about the new v2 API output, then the JSON is not really conducive to loading as single tweets as the JSON object is quite complex with bits of the information spread across multiple places (i.e. not all nicely nested to give one object per tweet).

I've implemented code that converts the v2 back to v1.1 as part of this private repo https://github.com/GateNLP/Twitter-API-v2 so that you can pull from the new API but use all the existing tools.

[johann-petrak]

Thanks - I already had the impression that v2 made this incredibly more complex but was not sure if maybe I was missing something.