Get rid of array job support

[PrometheusPi]

Dear users,

is anybody relying on the array job support? I have not used it for years and I would be very happy to rely entirely on MPI. This would allow a more lightweight code and setup routine and additionally would support getting rid of process_data and using direct MPI gather instead of storing spectra for each trajectory and disk.

The only advantage of array jobs is that they might start faster an very small clusters that are overbooked.
@TheresaBruemmer Would this be the case for Maxwell?
On Hypnos, Taurus and PizDaint this would not be an issue.
@QJohn2017 I assume since you are using the second largest cluster in the world, it is not an issue for you too. Is this correct?

[TheresaBruemmer]

What/where is the array job support? I don't think I ever used it...

[PrometheusPi]

Array jobs are jobs that are a selection of very similar single core jobs that get an ID from the submit system. With this ID each of the jobs can perform a different task, similar to the MPI rank distributed by MPI. However no cross communication is possible.

The advantage is, that since each job is a single core job, these jobs can start as soon as a single core is available. They can increase the occupancy of a cluster, but they also put a heavy load on the scheduler.

In contrast, an MPI job is a single job on multiple cores/nodes for the scheduler. Its scheduling is not a lot of load for the scheduler. But it might take some time till enough nodes are available, so that the job can start.

[TheresaBruemmer]

Well, I know the concept of array jobs. I did not know what you meant by array job support, but I guess you mean the compatibility of a clara2 run with splitting it into several array jobs.

In that case, I think it is ok to just include the ./process_data in the submit.sh
By that, clara2 can still be used by people who require this array job support.