Skip to content

Generate unique WARC file names when creating WACZ files #2

Open
@ibnesayeed

Description

@ibnesayeed

I was looking at WACZ files generated from ReplayWeb.page and noticed that the WARC file under the archive/ folder is always named data.warc.gz (or data.warc). It looks like the file name is hard-coded.

this.warcName = this.gzip ? "data.warc.gz" : "data.warc";

Would it be practical to add datetime or some other pseudo-unique or unique strings (like a hash or GUID) to the name (e.g., data-YYYYMMDDhhmmss.warc.gz)? One issue I can see is that if the same data is downloaded multiple times, a new timestamp might be added, making the WACZ file bytes non-reproducible, unless we use some deterministic identifier (like the datetime of the first capture or the hash of the WARC file itself, which might require renaming the file after writing data).

If people were to extract WARC files from more than one WACZ files and place them in a single folder, there is a chance of overwriting due to name collisions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions