Snellius mag

[vdejager]

---

name: New Config for Snellius HPC (Dutch National Supercomputer faclility
about: A new cluster config for both Snellius in general and one config for the 'mag' pipeline

Please follow these steps before submitting your PR:

• If your PR is a work in progress, include [WIP] in its title
• Your PR targets the master branch
• You've included links to relevant issues, if any

Steps for adding a new config profile:

• Add your custom config file to the conf/ directory
• Add your documentation file to the docs/ directory
• Add your custom profile to the nfcore_custom.config file in the top-level directory
• Add your custom profile to the README.md file in the top-level directory
• Add your profile name to the profile: scope in .github/workflows/main.yml
• OPTIONAL: Add your custom profile path and GitHub user name to .github/CODEOWNERS (**/<custom-profile>** @<github-username>)

[pontus]

Suggested change

//n = "${task.name}" -> n.replaceAll('[\\\s:()]', '_')

Seems like trial and error? (I'm not sure why you can't just do "${task.name}".replaceAll('[^a-zA-Z0-9]', '_').

[vdejager]

I'll see if that works. The above was suggested by the snellius admins

[pontus]

28 Gbyte for the nextflow monitoring process is really a lot. I haven't run mag but would be surprised if anywhere near that is needed.

[vdejager]

no, its not needed perse, but this is the smallest job one can specify on the cluster

[pontus]

Ok, I read the comment as the smallest that gets 28 Gbyte of RAM. Maybe clarify that's the minimum job size?

But that is still confusing since there are smaller requests used, do they still work and are silently scaled up? Since the expected task attempt handling of throwing more resources won't work as expected if they still get the same, maybe they should be adjusted then?

[pontus]

Maybe remove time when it's not changing the default?

[vdejager]

Maybe. In between I've been pointed at the scheduler strategy, so maybe I should make the default short. The mag pipeline typically runs longer per task (and 5 days is sometimes too short, so restarting the pipeline is needed)

[pontus]

In general, setting anything other than the maximum allowed doesn't make sense - the occasional earlier scheduling by backfill doesn't make up for what's wasted by jobs running fine but with too short time.

[pontus]

I naturally don't know anything about your system, but the two main reasons to play with time is either fit in what's allowed for jobs to a specific partition or possibly to get squeezed in by the backfill (and in some very specific cases not being stopped because the job as submitted would be more than the total allocation).

Are you sure these give any benefit and just not make things needlessly complicated (with a not-insignificant risk of well functioning jobs timing out when they wouldn't need to)?

[pontus]

Using https://www.nextflow.io/docs/latest/reference/process.html#resourcelimits is highly recommended.

[jfy133]

@nf-core-bot fix linting

[jfy133]

pre-commit linnitng error:

docs/pipeline/mag/snellius.md:
	69: Wrong amount of left-padding spaces(want multiple of 2)
	70: Wrong amount of left-padding spaces(want multiple of 2)
	71: Wrong amount of left-padding spaces(want multiple of 2)
	72: Wrong amount of left-padding spaces(want multiple of 2)
	73: Wrong amount of left-padding spaces(want multiple of 2)
	74: Wrong amount of left-padding spaces(want multiple of 2)
	75: Wrong amount of left-padding spaces(want multiple of 2)
	76: Wrong amount of left-padding spaces(want multiple of 2)
	```

[jfy133]

And tests aren't working because you haven't got all files for both the global and pipeline specific configs:

1. Global profile: https://github.com/nf-core/configs?tab=readme-ov-file#uploading-to-nf-coreconfigs
2. Pipeline profile: https://github.com/nf-core/configs?tab=readme-ov-file#adding-a-new-pipeline-specific-config

The specific error is:

Tests don't seem to test these profiles properly. Please check whether you added the profile to the Github Actions testing YAML.

{'snellius'}