Scheduled replications don't work after NTP issue #22470
Description
If you are reporting a problem, please make sure the following information are provided:
Expected behavior and actual behavior:
A clear and concise description of what you expected to happen and what's the actual behavior. If applicable, add screenshots to help explain your problem.
At some point in time we had issues with NTP in our VMs and they showed actual date 2028 year. After fixing it we noticed that scheduled pull replication tasks of Harbor in this particular VM didn't work. In Harbor UI we saw that tasks have Start Time 2028 year. So we tried to delete replication rules. Deletion was partially successful. Some of rules returned errors like this The execution 361082 has tasks that aren't in final status, stop the tasks first.
With the help of this [comment] (#8202 (comment)) we deleted stuck executions in postgresql and recreated replication rules via terraform. Currently manual trigger does work, but scheduled trigger is still broken.
Where should we look? Which service triggers jobservice? I checked harbor-core logs, but they are seemed ok to me. What else do we need to clear somewhere to restore scheduled replications?
Steps to reproduce the problem:
Not sure that I can reproduce problem.
Versions:
- harbor version: [2.11.1]
- docker engine version: [27.3.1]
- docker-compose version: [2.29.7]
Additional context:
- Harbor config files: The only changes were done in the file are - hostname, https.certificate, https.private_key, harbor_admin_password, database.password.
- Log files:harbor-jobservice.log