Marie Kondo'ing the bucket

[leothomas]

Detections sub-folders should follow the convention: detections-<type> (eg: detections-plane).

TODO:

• Copy all (new) data from detection-contrails/ to detections-contrail/
• rename detections-vehicles/ to detections-vehicle/
• remove wq-greatlakes-sm/ and wq-greatlakes-chl/ (these were renamed to wq-gl-spm/ and wq-gl-chl/ respectively (the spotlight id gl is necessary for the dates to pass the filtering, since the filenames themselves don't contain the spotlight id).
• remove /CropMonitor-* from agriculture/ (these files were moved from agriculture/ to agriculture-cropmonitor/

There's a folder called airport that has data from Nov 2019 to June 2020 for spotlight tk. Is that related to detections-plane ?

[olafveerman]

@leothomas Can you hold off from removing the CropMonitor files? Have to double check it's not used somewhere.

[leothomas]

Update:

With the updated Nightlights dataset the following folders can be deleted once the updated dataset is validated in production.

The backups from the previous nightlights dataset deployment:

• s3://covid-eo-blackmarble/BMHD_30M_MONTHLY_bkp
• s3://covid-eo-blackmarble/BM_500M_DAILY_bkp

The staging folders from the current nightlights dataset deployment:

• s3://covid-eo-data/bmhd_30m_monthly_staging
• s3://covid-eo-data/bm_500m_daily_staging