Skip to content

Resurrect janitor #8465

@aarongable

Description

@aarongable

In ages long past, we deleted boulder-janitor and replaced it with partman, a MariaDB partition-based system for deleting old data. With the move to Vitess, it is likely that partman will no longer work as it currently does.

We'd like to bring back something that looks vaguely like the janitor, but does a much better job of deleting data without causing database lock contention and replication storms. The result should run successfully against both ProxySql+MariaDB and Vitess+MySQL.

V0 of this project looks like:

  1. Pick a table (e.g. authz2) that we know is large in prod
  2. Create a large synthetic version of this table (maybe using the load generator)
  3. With the load generator still running, run the janitor at 2x the rate of row creation
  4. Maybe do that again, with replication set up between two VMs/containers

This could be done in a smaller project, with a fast custom row inserter, without having to worry about Boulder's whole scaffolding and schema.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions