Skip to content

New Scraper Framework #68

@njall

Description

@njall

Recreate the Ruby scraper library in NodeJS.

The motivation is to broaden the ability of 3rd parties to contribute scrapers by using a more ubiquitous language.

To recreate the scraper framework you'll need to make a new API client that interfaces with the write API. The Ruby version is kept as a separate library here: https://github.com/ElixirTeSS/TeSS_api_client

Then you'll have to produce a common utility library (see Scraper modules in https://github.com/ElixirTeSS/TeSS_scrapers/blob/master/lib/tess/scrapers/scraper.rb). In Ruby, the scrapers inherit from this module which interfaces with the TeSS API and collects statistics about success rates.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions