💡[feat] delete the task logs

[KyanChen]

Describe the problem

how to delete the shell task logs?

Describe the solution you'd like

how to delete the shell task logs?

Describe alternatives you've considered

No response

Additional context

No response

[ioga]

we don't have an explicit method to clean up these. they automatically go away after some time (72 hours if I remember corretcly).

[stoksc]

There is a relatively new master configuration that looks like this:

retention_policy:
  log_retention_days: 7
  schedule: 0 0 * * *

This will retain logs for 7 days and enforce retention with a GC job every night at midnight. Experiments and trials are able to overwrite this cluster-wide default by passing:

retention_policy:
  log_retention_days: -1

The -1 causes us to retain logs indefinitely.

[stoksc]

There is also a corresponding det task cleanup-logs command that will enforce the retention policy right away, but this still runs for all tasks, unfortunately.