Skip to content

Commit 77e7bd9

Browse files
cdkharriscdkharris
authored
Merge pull request #36 from UCL-ARC/cdkharris/snippets-updates
feat: Update the instructor notes for Fall 2024
2 parents 901d999 + b423a48 commit 77e7bd9

File tree

8 files changed

+81
-138
lines changed

8 files changed

+81
-138
lines changed

‎_config.yml

+1-2
Original file line numberDiff line numberDiff line change
@@ -29,10 +29,9 @@ local:
2929

3030
remote:
3131
name: "Myriad"
32-
host: "myriad.ucl.ac.uk"
3332
login: "myriad.rc.ucl.ac.uk"
3433
host: "login12.myriad.ucl.ac.uk"
35-
node: node-d00a-001
34+
node: node-d00a-005
3635
location: "University College London"
3736
homedir: "/home"
3837
user: "yourUsername"

‎_episodes/13-scheduler.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -218,7 +218,7 @@ later episode of this lesson.
218218
> >
219219
> > ```
220220
> > {{ site.remote.bash_shebang }}
221-
> > {{ site.sched.comment }} {{ site.sched.flag.time }}00:01:00 # timeout in HH:MM
221+
> > {{ site.sched.comment }} {{ site.sched.flag.time }}00:01:00 # timeout in HH:MM:SS
222222
> >
223223
> > echo -n "This script is running on "
224224
> > sleep 20 # time in seconds
@@ -247,7 +247,7 @@ wall time, and attempt to run a job for two minutes.
247247
```
248248
{{ site.remote.bash_shebang }}
249249
{{ site.sched.comment }} {{ site.sched.flag.name }} long_job
250-
{{ site.sched.comment }} {{ site.sched.flag.time }} 00:01 # timeout in HH:MM
250+
{{ site.sched.comment }} {{ site.sched.flag.time }}00:01:00 # timeout in HH:MM:SS
251251

252252
echo "This script is running on ... "
253253
sleep 240 # time in seconds

‎_episodes/15-modules.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -195,9 +195,9 @@ Let's examine the output of `module avail` more closely.
195195
> >
196196
> > ```
197197
> > {{ site.remote.bash_shebang }}
198-
> > {{ site.sched.comment }} {{ site.sched.flag.partition }}{% if site.sched.flag.qos %}
198+
> > {% if site.sched.flag.qos %}
199199
> > {{ site.sched.comment }} {{ site.sched.flag.qos }}
200-
> > {% endif %}{{ site.sched.comment }} {{ site.sched.flag.time }} 00:00:30
200+
> > {% endif %}{{ site.sched.comment }} {{ site.sched.flag.time }}00:00:30
201201
> >
202202
> > module load {{ site.remote.module_python3 }}
203203
> >

‎_episodes/17-parallel.md

+7-1
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,14 @@ Move into the extracted directory, then use the Package Installer for Python,
3838
or `pip`, to install it in your ("user") home directory:
3939

4040
```
41-
{{ site.remote.prompt }} cd amdahl
41+
{{ site.remote.prompt }} wget -O amdahl.tar.gz https://github.com/hpc-carpentry/amdahl/tarball/main
42+
{{ site.remote.prompt }} tar -xvzf amdahl.tar.gz
43+
{{ site.remote.prompt }} cd hpc-carpentry-amdahl-46c9b4b
44+
{{ site.remote.prompt }} module load python3/recommended
45+
{{ site.remote.prompt }} module unload compilers mpi
46+
{{ site.remote.prompt }} module load mpi4py
4247
{{ site.remote.prompt }} python3 -m pip install --user .
48+
{{ site.remote.prompt }} which amdahl
4349
```
4450
{: .language-bash}
4551

‎_episodes/18-resources.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ get your job dispatched earlier.
113113
> > finish within 2 minutes:
114114
> >
115115
> > ```
116-
> > {{ site.sched.comment }} {{ site.sched.flag.time }}{% if site.sched.name == "Slurm" %} {% else %}={% endif %}00:02:00
116+
> > {{ site.sched.comment }} {{ site.sched.flag.time }}00:02:00
117117
> > ```
118118
> > {: .language-bash}
119119
> {: .solution}

‎_includes/snippets_library/UCL_Myriad_sge/modules/software-dependencies.snip

+47-38
Original file line numberDiff line numberDiff line change
@@ -1,67 +1,72 @@
1-
To demonstrate, let's load the `ansys` module and then use the `module list`
2-
command to show which modules we currently have loaded in our environment.
3-
([ANSYS](https://www.ansys.com/en-gb) is an engineering simulation product.)
1+
To demonstrate, let's use `module list`. `module list` shows all loaded
2+
software modules.
43

54
```
6-
{{ site.remote.prompt }} module load ansys
5+
{{ site.remote.prompt }} module list
76
```
87
{: .language-bash}
98

109
```
11-
ansys/2019.r3(73):ERROR:151: Module 'ansys/2019.r3' depends on one of the
12-
module(s) 'giflib/5.1.1'
13-
ansys/2019.r3(73):ERROR:102: Tcl command execution failed:
14-
prereq giflib/5.1.1
10+
Currently Loaded Modulefiles:
11+
1) gcc-libs/4.9.2 10) nano/2.4.2 19) compilers/intel/2018/update3
12+
2) cmake/3.21.1 11) nedit/5.6-aug15 20) mpi/intel/2018/update3/intel
13+
3) flex/2.5.39 12) dos2unix/7.3 21) default-modules/2018
14+
4) git/2.32.0 13) giflib/5.1.1 22) python/3.9.10
15+
5) apr/1.7.0 14) emacs/28.1 23) openblas/0.3.7-serial/gnu-4.9.2
16+
6) apr-util/1.6.1 15) tmux/3.3a 24) python3/3.9
17+
7) subversion/1.14.1 16) mrxvt/0.5.4 25) python3/recommended
18+
8) screen/4.9.0 17) userscripts/1.4.0
19+
9) gerun 18) rcps-core/1.0.0
1520
```
1621
{: .output}
1722

18-
This shows that the default `ansys` module will not run because it first needs
19-
`giflib/5.1.1` to be loaded. Some HPC systems will automatically load
20-
dependencies like this, but at the time of writing (June 2020) UCL's Myriad
21-
does not.
22-
23-
Let's load the `giflib` module:
24-
2523
```
26-
{{ site.remote.prompt }} module load giflib
24+
{{ site.remote.prompt }} module load matlab
25+
{{ site.remote.prompt }} module list
2726
```
2827
{: .language-bash}
2928

3029
```
31-
giflib/5.1.1(18):ERROR:151: Module 'giflib/5.1.1' depends on one of the
32-
module(s) 'gcc-libs/4.9.2'
33-
giflib/5.1.1(18):ERROR:102: Tcl command execution failed: prereq gcc-libs
30+
Currently Loaded Modulefiles:
31+
1) gcc-libs/4.9.2 10) nano/2.4.2 19) compilers/intel/2018/update3
32+
2) cmake/3.21.1 11) nedit/5.6-aug15 20) mpi/intel/2018/update3/intel
33+
3) flex/2.5.39 12) dos2unix/7.3 21) default-modules/2018
34+
4) git/2.32.0 13) giflib/5.1.1 22) python/3.9.10
35+
5) apr/1.7.0 14) emacs/28.1 23) openblas/0.3.7-serial/gnu-4.9.2
36+
6) apr-util/1.6.1 15) tmux/3.3a 24) python3/3.9
37+
7) subversion/1.14.1 16) mrxvt/0.5.4 25) python3/recommended
38+
8) screen/4.9.0 17) userscripts/1.4.0 26) xorg-utils/X11R7.7
39+
9) gerun 18) rcps-core/1.0.0 27) matlab/full/r2023a/9.14
3440
```
3541
{: .output}
3642

37-
Here, we see that the `giflib` module itself also has a dependency, `gcc-libs`.
38-
So we have to load that first, then load `giflib`, and then finally load
39-
`ansys`.
43+
So in this case, loading the `matlab` module also loaded `xorg-utils/X11R7.7`, software
44+
for supporting a graphic user interface. Let's try unloading the `matlab` package.
4045

4146
```
42-
{{ site.remote.prompt }} module load gcc-libs/4.9.2
43-
{{ site.remote.prompt }} module load giflib/5.1.1
44-
{{ site.remote.prompt }} module load ansys
47+
{{ site.remote.prompt }} module unload matlab
48+
{{ site.remote.prompt }} module list
4549
```
4650
{: .language-bash}
4751

4852
```
49-
~/Scratch/.config is configured
50-
...
51-
...
52-
~/.mw doesn't exist - creating
53+
Currently Loaded Modulefiles:
54+
1) gcc-libs/4.9.2 10) nano/2.4.2 19) compilers/intel/2018/update3
55+
2) cmake/3.21.1 11) nedit/5.6-aug15 20) mpi/intel/2018/update3/intel
56+
3) flex/2.5.39 12) dos2unix/7.3 21) default-modules/2018
57+
4) git/2.32.0 13) giflib/5.1.1 22) python/3.9.10
58+
5) apr/1.7.0 14) emacs/28.1 23) openblas/0.3.7-serial/gnu-4.9.2
59+
6) apr-util/1.6.1 15) tmux/3.3a 24) python3/3.9
60+
7) subversion/1.14.1 16) mrxvt/0.5.4 25) python3/recommended
61+
8) screen/4.9.0 17) userscripts/1.4.0
62+
9) gerun 18) rcps-core/1.0.0
5363
```
5464
{: .output}
5565

56-
If you now use the `module list` command, you should see these three modules
57-
included in the list.
58-
59-
To unload a specific module, e.g. `ansys`, run the command
60-
`module unload ansys`. (On some systems, this will also unload the modules it
61-
depends on. Currently this is not the case with Myriad.)
62-
63-
If we wanted to unload everything at once (all modules), we could run
64-
`module purge` (unloads everything).
66+
So using `module unload` "un-loads" a module, and depending on how a site is
67+
configured it may also unload all of the dependencies (in our case it does
68+
not). If we wanted to unload everything at once, we could run `module purge`
69+
(unloads everything).
6570

6671
```
6772
{{ site.remote.prompt }} module purge
@@ -73,3 +78,7 @@ If we wanted to unload everything at once (all modules), we could run
7378
No Modulefiles Currently Loaded.
7479
```
7580
{: .output}
81+
82+
Note that `module purge` is informative. It will also let us know if a default
83+
set of "sticky" packages cannot be unloaded (and how to actually unload these
84+
if we truly so desired).

‎_includes/snippets_library/UCL_Myriad_sge/modules/wrong-gcc-version.snip

+20-91
Original file line numberDiff line numberDiff line change
@@ -45,120 +45,53 @@ matlab and see what we get.
4545
{: .language-bash}
4646

4747
```
48-
matlab/full/r2019b/9.7(99):ERROR:151: Module 'matlab/full/r2019b/9.7'
49-
depends on one of the module(s) 'gcc-libs/4.9.2'
50-
matlab/full/r2019b/9.7(99):ERROR:102: Tcl command execution failed:
51-
prereq gcc-libs
52-
```
53-
{: .output}
54-
55-
Here, we see that the default version of Matlab on the system is r2019b/9.7,
56-
which in this case is the most recent version. However, you should not assume
57-
that the default version is necessarily the latest.
58-
59-
As we saw in the earlier example, there are one or more dependencies.
60-
61-
Suppose we decide to load an earlier version of Matlab, e.g. r2017a/9.2.
62-
63-
```
64-
{{ site.remote.prompt }} module purge
65-
{{ site.remote.prompt }} module list
66-
```
67-
{: .language-bash}
68-
69-
```
70-
No Modulefiles Currently Loaded.
71-
```
72-
{: .output}
73-
74-
```
75-
{{ site.remote.prompt }} module load matlab/full/r2017a/9.2
76-
```
77-
{: .language-bash}
78-
79-
```
80-
matlab/full/r2017a/9.2(96):ERROR:151: Module 'matlab/full/r2017a/9.2' depends
81-
on one of the module(s) 'gcc-libs/4.9.2'
82-
matlab/full/r2017a/9.2(96):ERROR:102: Tcl command execution failed:
83-
prereq gcc-libs
84-
```
85-
{: .output}
8648
87-
```
88-
{{ site.remote.prompt }} module load gcc-libs/4.9.2
89-
{{ site.remote.prompt }} module load matlab/full/r2017a/9.2
90-
```
91-
{: .language-bash}
92-
93-
```
94-
matlab/full/r2017a/9.2(97):ERROR:151: Module 'matlab/full/r2017a/9.2' depends
95-
on one of the module(s) 'xorg-utils/X11R7.7'
96-
matlab/full/r2017a/9.2(97):ERROR:102: Tcl command execution failed:
97-
prereq xorg-utils/X11R7.7
98-
```
99-
{: .output}
100-
101-
```
102-
{{ site.remote.prompt }} module load xorg-utils/X11R7.7
103-
{{ site.remote.prompt }} module load matlab/full/r2017a/9.2
104-
```
105-
{: .language-bash}
106-
107-
```
10849
~/.matlab is a symbolic link pointing to /home/yourUsername/Scratch/.matlab
10950
11051
Matlab setup complete type matlab to start Matlab.
111-
```
112-
{: .output}
11352
114-
```
115-
{{ site.remote.prompt }} matlab -nodisplay -nosplash -nodesktop
116-
```
117-
{: .language-bash}
11853
54+
Loading matlab/full/r2023a/9.14
55+
Loading requirement: gcc-libs/10.2.0 xorg-utils/X11R7.7
11956
```
120-
< M A T L A B (R) >
121-
Copyright 1984-2017 The MathWorks, Inc.
122-
R2017a (9.2.0.556344) 64-bit (glnxa64)
123-
March 27, 2017
57+
{: .output}
12458

125-
To get started, type one of these: helpwin, helpdesk, or demo.
126-
For product information, visit www.mathworks.com.
59+
Here, we see that the default version of Matlab on the system is `r2023a/9.14`,
60+
which in this case is the most recent version. However, you should not assume
61+
that the default version is necessarily the latest.
12762

128-
>> quit
129-
```
130-
{: .output}
63+
As we saw in the earlier example, there are one or more dependencies.
13164

132-
Note that you cannot load two different versions of the same software at once.
133-
Currently, we have loaded `matlab/full/r2017a/9.2`. Let's try also loading
134-
`matlab/full/r2015b/8.6`:
65+
Suppose we decide to load an earlier version of Matlab, e.g. r2017a/9.2.
66+
You cannot load two different versions of the same software at once.
67+
Currently, we have loaded `matlab/full/r2023a/9.14`. Let's try also loading
68+
`matlab/full/r2021a/9.10`:
13569

13670
```
137-
{{ site.remote.prompt }} module load matlab/full/r2015b/8.6
71+
{{ site.remote.prompt }} module load matlab/full/r2021a/9.10
13872
```
13973
{: .language-bash}
14074

14175
```
142-
matlab/full/r2015b/8.6(108):ERROR:150: Module 'matlab/full/r2015b/8.6'
143-
conflicts with the currently loaded module(s) 'matlab/full/r2017a/9.2'
144-
matlab/full/r2015b/8.6(108):ERROR:102: Tcl command execution failed:
145-
conflict matlab
76+
Loading matlab/full/r2021a/9.10
77+
ERROR: Module cannot be loaded due to a conflict.
78+
HINT: Might try "module unload matlab" first.
14679
```
14780
{: .output}
14881

14982
As we can see, we get an error message about conflicts.
150-
If we do indeed wish to load version r2015b/8.6, we can say
83+
If we do indeed wish to load version `r2021a/9.10`, we can say
15184

15285
```
153-
{{ site.remote.prompt }} module unload matlab/full/r2017a/9.2
154-
{{ site.remote.prompt }} module load matlab/full/r2015b/8.6
86+
{{ site.remote.prompt }} module unload matlab
87+
{{ site.remote.prompt }} module load matlab/full/r2021a/9.10
15588
```
15689
{: .language-bash}
15790

15891
or, in one step:
15992

16093
```
161-
{{ site.remote.prompt }} module swap matlab matlab/full/r2015b/8.6
94+
{{ site.remote.prompt }} module swap matlab/full/r2021a/9.10
16295
```
16396
{: .language-bash}
16497

@@ -171,10 +104,6 @@ Check that this module has been loaded:
171104

172105
```
173106
Currently Loaded Modulefiles:
174-
1) gcc-libs/4.9.2 9) gerun 17) userscripts/1.4.0
175-
... ... ...
176-
6) apr-util/1.5.4 14) emacs/24.5 22) xorg-utils/X11R7.7
177-
7) subversion/1.8.13 15) tmux/2.2 23) matlab/full/r2015b/8.6
178-
8) screen/4.2.1 16) mrxvt/0.5.4
107+
1) gcc-libs/10.2.0 2) xorg-utils/X11R7.7 3) matlab/full/r2021a/9.10
179108
```
180109
{: .output}

‎_includes/snippets_library/UCL_Myriad_sge/scheduler/print-sched-variables.snip

+1-1
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
> >
1919
> > ```
2020
> > {{ site.remote.bash_shebang }}
21-
> > #SGE -l 00:00:30
21+
> > {{ site.sched.comment }} {{ site.sched.flag.time}}00:00:30
2222
> >
2323
> > echo -n "This script is running on "
2424
> > hostname

0 commit comments

Comments
 (0)