More and more frequently we need to backport certain PRs to the previous-major branch. We should automate this.