Submitting a Job for Cloud Scheduler

Submitting a job for use with Cloud Scheduler is very similar to submitting a job for use with a regular Condor Scheduler. It would be helpful to read through Chapter 2 of the Condor Manual for help on submitting jobs to Condor.

Jobs meant to be run by VMs started by Cloud Scheduler need a few extra parameters to work properly. These are (Required parameters are in bold):

• Requirements = VMType =?= “your.vm.type” : The type of VM that the job must run on. This is a custom attribute of the VM advertised to the Condor central manager. It should be specified on the VM’s condor_config or condor_config.local file.
• VMLoc or VMAMI : The URL (for Nimbus) or AMI (for EC2-like clusters) of the image required for the job to run
• VMCPUArch : The CPU architecture that the job requires. x86 or x86_64. Defaults to x86.
• VMCPUCores : The number of CPU cores for the VM. Defaults to 1.
• VMStorage : The amount of scratch storage space the job requires in GB. (Currently ignored on EC2-like Clusters)
• VMMem : The amount of RAM that the VM requires.
• VMNetwork : The type of networking required for your VM. Only used with Nimbus. Corresponds to Nimbus’s network pool.
• VMInstanceType : The EC2 instance type of the VM requested. Only used with EC2 clouds like Amazon.
• VMMaximumPrice : The maximum price in cents per hour for a VM (EC2 Only)
• VMKeepAlive : Number of minutes a VM should stay up after job finishes
• VMHighPriority : 1 (Optional flag) Indicates a high priority job to Cloud Scheduler – high priority job support can be enabled in the cloud_scheduler.conf
• TargetClouds : A comma separated list of names of clouds that you would like your job to use
• CSMyProxyServer : The hostname of the myproxy server you’d like to use for credential renewal
• CSMyProxyCredsName : The name of your myproxy credentials
• VMJobPerCore : bool – Assigns multiple slots to a multi-core VM

A sample job

# Regular Condor Attributes
Universe   = vanilla
Executable = script.sh
Arguments  = one two three
Log        = script.log
Output     = script.out
Error      = script.error
should_transfer_files = YES
when_to_transfer_output = ON_EXIT
# 
# Cloud Scheduler Attributes
Requirements = VMType =?= "vm.for.script"
+VMLoc         = "http://repository.tld/your.vm.img.gz"
+VMAMI = "ami-dfasfds"
+VMCPUArch     = "x86"
+VMCPUCores    = "1"
+VMNetwork     = "private"
+VMMem         = "512"
+VMStorage     = "20"
Queue