These tests are not enabled in CI. As proposed in #404 we will have a new tests/smoke suite that is written in pytest. It's already triggered in CI, so we should migrate the existing scenarios to the new framework and then remove the shell script.