TitanPackaging

Background

This page documents a sub-task of the HPC adaptation project at UiO; please see the TitanTop page for background.

For the first few weeks of the project, participants from both groups will establish a shared knowledge of the relevant DELPH-IN tools, standard tasks and use patterns, and their interaction with the HPC environment. This work package will be organized as a series of hands-on 'walk-through' sessions, most likely at a rate of about one half day per week. The main result of this phase in the project will be a mutual, in-depth understanding of the available resources, requirements, and limitations in current software versions. This knowledge will be documented through a collection of collaboratively authored wiki pags.

Based on the observations made in the initial project phase, VD and user staff will jointly adapt and 'package' DELPH-IN tools for batch processing, i.e. prepare a gradual shift of use patterns, away from the currently predominant interactive style. This work package will create a standard set of job scripts and configuration mechanisms that automate common tasks and provide a good balance between ease of use and flexibility, i.e. user customization of relevant parameters. The resources developed in this work package will be contributed as new open-source components to the DELPH-IN repository.

TITAN-Specific Files

There is a preliminary collection of TITAN-specific files, including a few (imperfect) SLURM job scripts in the sub-directory uio/titan/ inside the LOGON tree.

Parsing

Parsing a corpus (i.e. a sequence of sentences) is an 'embarassingly parallel' problem, in the sense that processing each sentence is completely independent of other sentences.

The itsdb controller (written predominantly in Lisp) supports parallelization (and distribution) across nodes at the sentence level, using PVM. The parser itself is part of the [http://www.delph-in.net/pet PET] package, implemented in C++ (with critical routines in pure C). A typical interactive parser invocation could be the following:

  $LOGONROOT/parse --erg+tnt --count 7 --best 500 cb

The parse command is a shell script that will (a) launch the controller; (b) load the grammar identified as erg+tnt (the English Resource Grammar, [http://www.delph-in.net/erg ERG], used in conjunction with the TnT tagger); (c) use PVM routines to create 7 PET client processes for parsing; (d) configure the parsing environment to return up to 500 most probable parses; and (e) work through the corpus identified as cb ([http://www.catb.org/esr/writings/cathedral-bazaar/cathedral-bazaar/ The Cathedral and the Bazaar]). The parse script and a few more existing LOGON scripts are discussed in more detail on the LogonProcessing pages.

By default, itsdb will launch a PVM daemon on the current node if necessary (i.e. if there is no existing daemon on that node for the current user). That means that putting seven PET clients on a single eight-core node is easy, as would be putting 31 such clients on a 32-core node. To take advantage of multiple nodes, however, PVM initialization will need to be informed of the set of nodes (and number of cores per node available). StephanOepen used to have half-baked scripted support to retrieve that information from the older SGE environment, then create a PVM initialization file (.pvm_hosts), and then ask itsdb to use all available PVM nodes. These steps should be adapted for SLURM, made robust (to a certain degree), and supported in standard scripts.

Questions and Answers

Where do I get feedback on why my jobs fail?

• My initial attempts at producing a 'parse' job script had erroneously specified --mem-per-cpu=2M; the jobs were started and terminated immediately, but even when requesting --mail-type=ALL, I did not see any indication of why my jobs were killed (nor did I find a log file providing that information).

How can I set common SLURM defaults?

• For use with the older SGE environment, I had a user configuration file with SGE defaults (project, maximum wall-clock time, mail notification, and so on). Is there a corresponding facility in SLURM? If not, is there a way to include #SBATCH statements near the top of each job file?

Can I pass command-line arguments into SLURM job scripts?