Skip to content

[Bug Report] yml scrapers aren't self-aware of url #4087

Open
@scruffynerf

Description

@scruffynerf

a yml scraper, despite being called against a url, doesn't have a way to directly return that url as a value,
depending entirely on the resulting page having some form of url content to return.
rl
For most bigger sites where a url is provided, this isn't an issue, but the lesser sites where you can't easily get a url of what you are scraping, this is a problem, and often the result is the scraper doesn't return a url, which can be a problem when doing tasks that expect a url value to be returned. I've seen this result in scraping and then emptying the url value, which means the scene can't easily be rescraped, for example.

What should happen:
some way to make the yml scraper return the called url AS the URL result, if not otherwise set.

Metadata

Metadata

Assignees

No one assigned

    Labels

    improvementSomething needed tweaking.

    Type

    No type

    Projects

    Status

    No status

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions