A Shipyard HPC Script task can execute a HPC job using Azure Batch Shipyard.
For a Shipyard HPC Script task, you will need:
You have the option of using either Docker or Singularity containers in a Shipyard HPC script task. You can specify which container you would like to use as part of the shipyard process in the field Container Type.
If you choose to use a Singularity container, you also have the option to provide a ‘Signing Key Fingerprint’.
You can specify a different nvidia driver for a different workloads using a new parameter.
You can create a parameter with the name
The value for this parameter would be the public URL of the Nvidia driver that you want to use.
You can also add a parameter to change the VM size used in Shipyard tasks. It will specify a different VM size, and if this parameter is present, this should override the VM size from the Azure Batch Cluster Definition.
Name your parameter
The value for this parameter should be the VM size of a VM found here.
You can copy a VM size value from the above link into Loome Integrate’s parameter value field.
In your chosen job, add a new task.
Select Shipyard HPC Script as the task type.
Select the Azure Batch connection from the dropdown.
Select the Azure Blob Storage connection from the dropdown.
Select your Cluster Definition from the dropdown.
Choose your Container Type from the dropdown. This is where you choose Docker or Singularity.
Enter the name of an image in your Container Registry.
If you chose Singularity as your Container Type, you can optionally provide a Signing Key Fingerprint.
Select your Container Registry Connection from the dropdown field.
Enter the script that should be run as part of your job.
You can optionally add storage via other Azure Storage accounts using the fields Shared Volume Connection and Volume File Definition.
You can then enter any parameters. To add a parameter, add a name and default value, and then click on the Add Parameter button on the right of the row.
Click Submit and it will save this task.
You can then run the Loome Integrate Job to run this Task.
You can monitor the logs by viewing the execution history of this Job.