Skip to content

Switch crawl bundle format #14

@ato

Description

@ato

The tarball crawl bundles Bamboo currently use make it expensive to retrieve the logs from for QA reporting and analysis. UKWA use zip64 crawl log/config bundles, might nice to use the same structure.

@anjackson pointed me at the relevant part of their code: https://github.com/ukwa/python-shepherd/blob/2016-pulse/crawl/job/output.py#L153-L175

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions