Shipyard HPC Script

A Shipyard HPC Script task can execute a HPC job using Azure Batch Shipyard.

Docker options for Shipyard HPC script

You may need to set up your connections, such as your container registry, on the connections page and a cluster definition before you can create and submit this task.

For a Shipyard HPC Script task, you will need:

  • Azure Batch Connection
  • Azure Blob Connection (Azure Storage connection)
  • Cluster Definition
  • Container Type (Docker/Singularity)
  • Container Image
  • Script

Optional fields:

  • Container Registry Connection: Providing a custom container registry will allow for the usage of Container images not available from the Docker Hub or other Container Libraries.
  • Signing Key Fingerprint: You can provide this if you have selected a Singularity container
  • Shared Volume Connection: Select an Azure Storage Connection. This allows for an Azure Blob or File Share resource to be mounted to the container as a shared volume.
  • Volume File Definition
  • Task Parameters

You have the option of using either Docker or Singularity containers in a Shipyard HPC script task. You can specify which container you would like to use as part of the shipyard process in the field Container Type.

If you choose to use a Singularity container, you also have the option to provide a ‘Signing Key Fingerprint’.

Singularity container and signing key fingerprint fields

How to Set up a Shipyard HPC Task

In your chosen job, add a new task.

Select Shipyard HPC Script as the task type.

Select the Azure Batch connection from the dropdown.

Select the Azure Blob Storage connection from the dropdown.

Select your Cluster Definition from the dropdown.

Choose your Container Type from the dropdown. This is where you choose Docker or Singularity.

Enter the name of an image in your Container Registry.

If you chose Singularity as your Container Type, you can optionally provide a Signing Key Fingerprint.

Select your Container Registry Connection from the dropdown field.

Enter the script that should be run as part of your job.

You can optionally add storage via other Azure Storage accounts using the fields Shared Volume Connection and Volume File Definition.

You can then enter any parameters. To add a parameter, add a name and default value, and then click on the Add Parameter button on the right of the row.

Click Submit and it will save this task.

You can then run the Loome Integrate Job to run this Task.

You can monitor the logs by viewing the execution history of this Job.