E2E upgrade tests (and maybe more tests) cannot be run locally

**Describe the bug**
As noted in https://github.com/knative/serving-operator/pull/300, e2e upgrade tests cannot be run locally.

**Expected behavior**
Should be runing locally fine.

**Additional context**
There were errors around Istio. I don't have the log messages handy.
