how to specify job specific configuration

[nh13]

I am looking to set batch job specific information prior to submission, for example:

• job definition (to allow for custom cloud init if needed)
• job queue
• instance types (may be achieved through a queue) for tasks that need GPU or FPGA
• disks to mount, and/or root/attached storage size

[jdidion]

Came here to ask for this same feature, specifically for instance types.

The WDL spec suggests that backend/specific configuration should be specified as follows:

WDL 1.x

runtime {
  aws: {
    instanceType: "c5.16xlarge"
  }
}

WDL development

hints {
  aws: {
    instanceType: "c5.16xlarge"
  }
}

[SebastianGuelfi]

Does anyone know how to configure the disk space in runtime? It seems "disks" is ignored by miniwdl and my workflow fails because "No space left on device" error.

[mlin]

@SebastianGuelfi

1. are you able to distinguish whether it's the task working directory (typically a shared NFS) vs. the container root filesystem (including /tmp; typically on local scratch disks or EBS volumes) that are filling up?
2. how are your Batch compute environments getting provisioned?