1 of 6

DRAGEN Multi-Cloud

DRAGEN on AWS

You can run DRAGEN analysis on Amazon Web Services (AWS). For information on using DRAGEN, see the DRAGEN User Guide Section. For information on using AWS, see the AMI documentation available on the Amazon Web Services site.

You must have access to a DRAGEN license to use DRAGEN on AWS. Contact Illumina Technical Support to obtain a DRAGEN license.

Launch an Instance

Navigate to the AWS Management Console, and then sign in.
Select the Services menu on the top toolbar, and then select EC2 under Compute.
Under Images, select AMIs.
Select the AMI ID of your instance.
Launch the image.
Select the instance type, and then select Next to configure additional VM settings. The f1.4xlarge instance type is recommended for DRAGEN. For more information on instance settings, see the AMI documentation available on the Amazon Web Services site.
After you have finished, select Review and Launch.
Select Choose an existing key pair or Proceed without a key pair, and then select Launch Instance.
When the launch confirmation page displays, select View Instances to navigate to your list of instances.

Configure Your Instance

Navigate to the Instances page, and then select the instance to connect to.
Select Connect.
Select SSH client and connect to the instance. For more information on connecting to an instance, see the AMI documentation available on the Amazon Web Services site.
Mount the disks to /staging on the instance using the following commands.

sudo yum -y install mdadm
sudo mdadm --create --verbose /dev/md0 --level=0 --name=MY_RAID0 --raid-devices=<number of volumes> <device name 1> <device name 2>
sudo mkfs.ext4 -L MY_RAID0 /dev/md0
sudo mkdir -p /staging
sudo sh -c "echo 'LABEL=MY_RAID0 /staging ext4 defaults,noatime 0 0' >> /etc/fstab"
sudo mount -a

For example, the following command mounts four volumes attached at /dev/xvdb, /dev/xvdc, /dev/xvdd, and /dev/xvde to a RAID 0 on /staging.

sudo yum -y install mdadm
sudo mdadm --create --verbose /dev/md0 --level=0 --name=MY_RAID0 --raid-devices=4 /dev/xvdb /dev/xvdc /dev/xvdd /dev/xvde
sudo mkfs.ext4 -L MY_RAID0 /dev/md0
sudo mkdir -p /staging
sudo sh -c "echo 'LABEL=MY_RAID0 /staging ext4 defaults,noatime 0 0' >> /etc/fstab"
sudo mount -a

You can use DRAGEN command-line options. For more information on DRAGEN analysis and command line options, see the Command Line Options Section of the user guide. For information on using AWS, see the AMI documentation available on the Amazon Web Services site.

DRAGEN on Microsoft Azure

You can run DRAGEN analysis on Microsoft Azure. For information on using DRAGEN, see the . For information on using Azure, see the Azure documentation available on the Microsoft site.

A QuickStart for DRAGEN on Azure is avaialble here

If you are using an Azure Batch account, see for more information.

Requirements

To use the DRAGEN Software for genomic data analysis on Azure, the following are required:

DRAGEN Multi-Cloud license. For more information, contact Illumina Technical Support.
Azure cloud subscription.
Quota for NP-series virtual machines (VM).
Azure command-line interface (CLI). For instructions on installing the Azure CLI, see the Azure documentation available on the Microsoft site.
Genomic data uploaded to your Azure Blob storage account.

Supported Regions

DRAGEN on Azure is available in the following regions with field programmable gate array (FPGA)-enabled, standard NP-series virtual machines.

West US 2
East US
South Central US
West Europe (Amsterdam)
Southeast Asia (Singapore)

Request a NP-series Quota

DRAGEN requires FPGA hardware. You must request access to the FPGA-enabled, NP-series VMs.

The vCPU requirements for NP-series SKUs are in increments of 10. When requesting an updated quota, DRAGEN recommends requesting vCPUs in batches of 10. The instructions for running DRAGEN on Azure require a minimum increase of 10 vCPU quotas for NP-series machines.

To make sure you have quota for NP-series VMs or to increase a quota, navigate to your Microsoft Azure portal, and then do as follows.

Select Subscriptions, then choose your subscription.
Under Settings, select Usage + quotas.
Enter NP into the search bar to filter for the NP-series. The quota list displays Standard NPS Family vCPUs.
If you do not see any results, select Request Increase. For more information on requesting quota for NP-series VMs, see the Azure documentation available on the Microsoft site.

Run DRAGEN VM on Azure

Use the following information to run the DRAGEN virtual machine (VM) on Microsoft Azure. For information on using Azure, see the Azure documentation available on the Microsoft site.

Navigate to the Microsoft Azure portal, and then sign in.
Select Marketplace.
Select View Private Offers, and then select DRAGEN on Azure.
Select Create. Starting from a preset configuration option is not recommended for DRAGEN.
Select a subscription and resource group from the drop-down menus, or select Create New.
Enter a name for the virtual machine.
Select a region that is compatible with the NP-series. See the Azure documentation available on the Microsoft site for more information.
Select DRAGEN and the current version as the image.
Select a storage size from the Size drop-down list. Only NP10 and NP20 sizes are compatible.
Configure any additional VM settings. For your disk type, DRAGEN recommends using Premium SSD for optimal performance. For information, see the Azure documentation available on the Microsoft site.
When finished, select Review + Create
To launch the VM, select Create.
After deployment completes, select Go to Resource. After your VM deploys, you can connect to the DRAGEN VM via the Azure Cloud Shell or another client of your choice. For more information on using a VM, see the Azure documentation available on the Microsoft site.

DRAGEN on Microsoft Azure Batch

You can run DRAGEN analysis on Microsoft Azure Batch. For information on using Azure, see the Azure documentation available on the Microsoft site.

If you are not using an Azure Batch account, see DRAGEN on Microsoft Azure for information on using DRAGEN without Azure Batch.

Requirements

To use DRAGEN on Azure, the following are required:

DRAGEN Multi-Cloud license. For more information, contact Illumina Technical Support.
Azure cloud subscription.
Quota for NP-series virtual machines (VM).
Azure command-line interface (CLI). For instructions on installing the Azure CLI, see the Azure documentation available on the Microsoft site.
Genomic data uploaded to your Azure Blob storage account.
The Azure directory principal must be assigned the Contributor role for the Azure subscription. If you would like to restrict access, you can run deployment pipelines as a managed service principal with the Contributor role.

Supported Regions

DRAGEN on Azure is available in the following regions with field programmable gate array (FPGA)-enabled, standard NP-series virtual machines.

West US 2
East US
South Central US
West Europe (Amsterdam)
Southeast Asia (Singapore)

Quota Requirements

Request a NP-series Quota

DRAGEN requires FPGA hardware. You must request access to the FPGA-enabled, NP-series VMs.

To make sure you have sufficient quota for NP-series VMs or to increase your quota, navigate to your Microsoft Azure portal, and then do as follows:

Select Subscriptions, then choose your subscription.
Under Settings, select Usage + quotas.
Enter NP into the search bar to filter for the NP-series. The quota list displays Standard NPS Family vCPUs.
If you do not see any results, select Request Increase. For more information on requesting quota for NP-series VMs, see the Azure documentation available on the Microsoft site.

Pool Allocation Modes

By default, Azure Batch allocates pools using the Batch service mode. In Batch service mode, compute nodes are separated into different quotas. To run DRAGEN in Batch service mode, you need to request additional quota for your Azure Batch account. For information on the current default quota for Batch accounts and on increasing your Azure Batch account quota, see the Azure Batch documentation available on the Microsoft Azure site.

If using user subscription mode for your Batch account, you must add the Azure Batch service as the Contributor role. For more information, see the Microsoft Azure Batch documentation available on the Microsoft site.

Azure Batch Run Modes

Run DRAGEN VM on Azure Batch

Use the following information to run the DRAGEN virtual machine (VM) on Microsoft Azure Batch. For information on using DRAGEN, see the see the . For information on using Azure, see the Azure documentation available on the Microsoft site.

Navigate to the Microsoft Azure portal, and then sign in.
Select Marketplace.
Select View Private Offers, and then select DRAGEN on Azure.
Select Create. Starting from a preset configuration option is not recommended for DRAGEN.
Select a subscription and resource group from the drop-down menus, or select Create New.
Enter a name for the virtual machine.
Select a region that is compatible with the NP-series. See the Azure documentation available on the Microsoft site for more information.
Select DRAGEN and the current version as the image.
Select a storage size from the Size drop-down list. Only NP10 and NP20 sizes are compatible.
Configure any additional VM settings. For your disk type, DRAGEN recommends using Premium SSD for optimal performance. For information, see the Azure documentation available on the Microsoft site.
When finished, select Review + Create
To launch the VM, select Create.
After deployment completes, select Go to Resource. After your VM deploys, you can connect to the DRAGEN VM via the Azure Cloud Shell or another client of your choice. For more information on using a VM, see the Azure documentation available on the Microsoft site. For more information on DRAGEN analysis and command line options,

Run DRAGEN With ARM

You can also run DRAGEN on Azure Batch using an Azure Resource Manager (ARM) template available on the DRAGEN Multi-Cloud support site.

The ARM template only includes the parameters required to run DRAGEN on Azure Batch. See the Microsoft Azure documentation available on the Microsoft site for information on configuring additional parameters.

Running DRAGEN using an ARM template enables the following advanced options.

Incorporating DRAGEN into an existing infrastructure.
Automating deployments with CI/CD pipelines.
Customizing the DRAGEN deployment.

Storage Account Parameters

The ARM template available on the DRAGEN Multi-Cloud support site creates a storage account and container. To use an existing Azure Blob storage account, specify the following input parameters in the ARM template.

storageNewOrExisting: existing
storageAccountName: <name of your existing storage account>

Run DRAGEN Using ARM Template

Use the following instructions to run DRAGEN on Azure Batch using the ARM template available on the DRAGEN Multi-Cloud support site.

Download the ARM template available on the DRAGEN Multi-Cloud support site.
Enter the following commands.

You can enter additional command line options to further customize the run, including maximum Batch job and task run time. See the Azure CLI documentation available on the Microsoft site for more information.

Use DRAGEN With the Azure Batch CLI

After creating and authenticating your Azure Batch account, use the following instructions to run DRAGEN with the Azure Batch CLI.

To run a DRAGEN Batch task, create a task.json file. The task.json file contains information on the Batch task, resource files, and output files. See Create the JSON file for information. You can then use the JSON file in the create Batch task command.

For more information on creating Azure Batch accounts and using the Azure Batch CLI, see the Azure Batch documentation available on the Microsoft site.

Create the JSON file

To set up the task.json to use in the Batch task create command, use the following structure.

Batch Task

Use the following instructions to configure the Batch task. For more information using the Azure Batch CLI, see the Azure Batch documentation available on the Microsoft site.

Create the batch job using the following command.

Enter the following command line to create the batch task. To use files located in a private Azure Blob storage account, see the Azure Blob CLI documentation available on the Microsoft site.

The following is an example Batch task configuration.

Resource Files

To add resource files to the Batch node, use the resourceFiles configuration in the task.json. The following example specifies the genome and FASTQ files. To use files located in a private Azure Blob storage account, see the Azure Blob CLI documentation available on the Microsoft site.

Output Files

To specify the location to write output files, use the outputFiles configuration in the task.json. The following command-line example places output logs and DRAGEN files in the specified storage container. To generate a storage container URL, use a SAS token. For more information on accessing a Blob storage container using a SAS token, see the Azure Blob CLI documentation available on the Microsoft site.

Example JSON File

The following is an example task.json file.

Stream Files

DRAGEN can stream input FASTQ and BAM files from private Azure Blob containers. The genome file must be located locally on the node. DRAGEN does not support streaming from public Blob containers.

Stream From Azure Blob Storage

Use the following command as the Batch task command. If using the following command, you do not need to specify resourceFiles in task.json.

Stream From FASTQ List

You can use a FASTQ list file to reference and stream FASTQ files. The FASTQ list file must be local to the node. The FASTQ files referenced in the FASTQ list can be URLs to files on a Blob storage account.

To configure resourceFiles to stream from a FASTQ file list file, use the following command. The FASTQ files in the following command are located on Blob storage account.

The task.json file is structured as follows.

Use the following command as the Batch task command. If using the following command, you do not need to specify resourceFiles in task.json.

Create the Batch Task

After you have set up the task.json file, you can use this file and Batch job ID to create the Azure Batch task with the following command.

For more information on creating Azure Batch tasks, see the Azure Batch documentation available on the Microsoft site.

Azure Batch Run Modes

Run DRAGEN VM on Azure Batch

Navigate to the Microsoft Azure portal, and then sign in.
Select Marketplace.
Select View Private Offers, and then select DRAGEN on Azure.
Select Create. Starting from a preset configuration option is not recommended for DRAGEN.
Select a subscription and resource group from the drop-down menus, or select Create New.
Enter a name for the virtual machine.
Select a region that is compatible with the NP-series. See the Azure documentation available on the Microsoft site for more information.
Select DRAGEN and the current version as the image.
Select a storage size from the Size drop-down list. Only NP10 and NP20 sizes are compatible.
Configure any additional VM settings. For your disk type, DRAGEN recommends using Premium SSD for optimal performance. For information, see the Azure documentation available on the Microsoft site.
When finished, select Review + Create
To launch the VM, select Create.
After deployment completes, select Go to Resource. After your VM deploys, you can connect to the DRAGEN VM via the Azure Cloud Shell or another client of your choice. For more information on using a VM, see the Azure documentation available on the Microsoft site. For more information on DRAGEN analysis and command line options,

Run DRAGEN With ARM

You can also run DRAGEN on Azure Batch using an Azure Resource Manager (ARM) template available on the DRAGEN Multi-Cloud support site.

Running DRAGEN using an ARM template enables the following advanced options.

Incorporating DRAGEN into an existing infrastructure.
Automating deployments with CI/CD pipelines.
Customizing the DRAGEN deployment.

Storage Account Parameters

storageNewOrExisting: existing
storageAccountName: <name of your existing storage account>

Run DRAGEN Using ARM Template

Use the following instructions to run DRAGEN on Azure Batch using the ARM template available on the DRAGEN Multi-Cloud support site.

Download the ARM template available on the DRAGEN Multi-Cloud support site.
Enter the following commands.

RESOURCE_GROUP="dragen"
az group create -n "$RESOURCE_GROUP" -l "EastUS"
az deployment group create \
-g "$RESOURCE_GROUP" \
-p prefix=fpgaci \
-p azureBatchServiceOid=795cc567-16b1-4904-9344-afc876387199 \
-f mainTemplate.json \
--query "properties.outputs"

Use DRAGEN With the Azure Batch CLI

After creating and authenticating your Azure Batch account, use the following instructions to run DRAGEN with the Azure Batch CLI.

For more information on creating Azure Batch accounts and using the Azure Batch CLI, see the Azure Batch documentation available on the Microsoft site.

Create the JSON file

To set up the task.json to use in the Batch task create command, use the following structure.

{
"id": "<Batch task ID. SeeBatch Task>",
"commandLine": "<Command line options>",
"resourcesFiles": [<Resource files. SeeResource Files>],
"outputFiles": [<Output file directory. SeeOutput Files.>]
}

Batch Task

Use the following instructions to configure the Batch task. For more information using the Azure Batch CLI, see the Azure Batch documentation available on the Microsoft site.

Create the batch job using the following command.

az batch job create --id <Unique ID for the job to create> pool-id <Name of the pool created with the ARM template>

Enter the following command line to create the batch task. To use files located in a private Azure Blob storage account, see the Azure Blob CLI documentation available on the Microsoft site.

/bin/bash -c \
"mkdir <Hash table reference directory> <Output directory>; \
tar xzvf dragen.tar -C <Hash table reference directory>; \
/opt/edico/bin/dragen --partial-reconfig HMM --ignore-version-check true; \
/opt/edico/bin/dragen -f -r <Hash table reference directory> \
-1 <Path to the first local FASTQ file on the node> \
-2 <Path to the second local FASTQ file on the node> \
--RGID <RGID associated with the DRAGEN run> \
--RGSM <RGSM associated with the DRAGEN run> \
--enable-bam-indexing true \
--enable-map-align-output true \
--enable-sort true \
--output-file-prefix dragen-batch \
--enable-map-align true \
--output-format BAM \
--output-directory <Output directory> \
--enable-variant-caller true \
--lic-server <DRAGEN license>"

The following is an example Batch task configuration.

/bin/bash -c \
"mkdir dragen output; \
tar xzvf dragen.tar -C dragen; \
/opt/edico/bin/dragen --partial-reconfig HMM --ignore-version-check true; \
/opt/edico/bin/dragen -f -r dragen \
-1 1.fq.gz \
-2 2.fq.gz \
--RGID NA24385-AJ-Son-R1-NS_S33 \
--RGSM NA24385-AJ-Son-R1-NS_S33 \
--enable-bam-indexing true \
--enable-map-align-output true \
--enable-sort true \
--output-file-prefix dragen-batch \
--enable-map-align true \
--output-format BAM \
--output-directory output \
--enable-variant-caller true \
--lic-server <LICENSE>"

Resource Files

"resourceFiles": [{
"filePath": "dragen.tar",
"httpUrl": "<URL to genome tarball file>"
}, {
"filePath": "<First FASTQ file name>.gz",
"httpUrl": "<URL to the first FASTQ file>"
}, {
"filePath": "<Second FASTQ file name>.gz",
"httpUrl": "<URL to the second FASTQ file>"
}]

Output Files

"outputFiles": [{
"filePattern": "../stdout.txt",
"destination": {
"container": {
"containerUrl": "<Container URL with SAS token>",
"path": "<Batch task ID to organize the output>/stdout.txt"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}, {
"filePattern": "../stderr.txt",
"destination": {
"container": {
"containerUrl": "<Container URL with SAS token>",
"path": "<Batch task ID to organize the output>/stderr.txt"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}, {
"filePattern": "<Directory to output results to>/**/*",
"destination": {
"container": {
"containerUrl": "<Container URL with SAS token>",
"path": "<Batch task ID to organize the output>/<Directory to output results to>"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}, {
"filePattern": "/var/log/dragen.log",
"destination": {
"container": {
"containerUrl": "<Container URL with SAS token>",
"path": "<Batch task ID to organize the output>/log/dragen.log"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}, {
"filePattern": "/var/log/dragen/**/*",
"destination": {
"container": {
"containerUrl": "<Container URL with SAS token>",
"path": "<Batch task ID to organize the output>/log/dragen"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}]

Example JSON File

The following is an example task.json file.

{
"id": "task1",
"commandLine": "$COMMAND",
"resourceFiles": [{
"filePath": "dragen.tar",
"httpUrl": "https://dragentestdata.blob.core.windows.net/reference-genomes/Hsapiens/hash-tables/hg38_altaware_nohla-cnv-anchored.v8.tar"
}, {
"filePath": "1.fq.gz",
"httpUrl": "https://dragentestdata.blob.core.windows.net/samples/wes/NA24385-AJ-Son-R1-NS_S33/NA24385-AJ-Son-R1-NS_S33_L001_R1_001.fastq.gz"
}, {
"filePath": "2.fq.gz",
"httpUrl": "https://dragentestdata.blob.core.windows.net/samples/wes/NA24385-AJ-Son-R1-NS_S33/NA24385-AJ-Son-R1-NS_S33_L001_R2_001.fastq.gz"
}],
"outputFiles": [{
"filePattern": "../stdout.txt",
"destination": {
"container": {
"containerUrl": "$CONTAINER_URL",
"path": "task1/stdout.txt"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}, {
"filePattern": "../stderr.txt",
"destination": {
"container": {
"containerUrl": "$CONTAINER_URL",
"path": "task1/stderr.txt"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}, {
"filePattern": "output/**/*",
"destination": {
"container": {
"containerUrl": "$CONTAINER_URL",
"path": "task1/output"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}, {
"filePattern": "/var/log/dragen.log",
"destination": {
"container": {
"containerUrl": "$CONTAINER_URL",
"path": "task1/log/dragen.log"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}, {
"filePattern": "/var/log/dragen/**/*",
"destination": {
"container": {
"containerUrl": "<CONTAINER_URL>",
"path": "task1/log/dragen"
}
},
"uploadOptions": {
"uploadCondition": "taskcompletion"
}
}]
}

Stream Files

DRAGEN can stream input FASTQ and BAM files from private Azure Blob containers. The genome file must be located locally on the node. DRAGEN does not support streaming from public Blob containers.

Stream From Azure Blob Storage

Use the following command as the Batch task command. If using the following command, you do not need to specify resourceFiles in task.json.

/bin/bash -c \
"echo DefaultEndpointsProtocol=https >> ~/.azure-credentials; \
echo AccountName=<Name of the Blob storage account> >> ~/.azure-credentials; \
echo AccountKey=<Access key to the Blob storage account> >> ~/.azure-credentials; \
echo EndpointSuffix=core.windows.net >> ~/.azure-credentials; \
mkdir dragen output; \
tar xzvf dragen.tar -C dragen; \
/opt/edico/bin/dragen --partial-reconfig HMM --ignore-version-check true; \
/opt/edico/bin/dragen -f -r dragen \
-1 <Full URL to the first FASTQ file in Blob storage> \
-2 <Full URL to the second FASTQ file in Blob storage> \
--RGID <RGID> \
--RGSM <RGSM> \
--enable-bam-indexing true \
--enable-map-align-output true \
--enable-sort true \
--output-file-prefix dragen-batch \
--enable-map-align true \
--output-format BAM \
--output-directory output \
--enable-variant-caller true \
--lic-server <LICENSE>"

Stream From FASTQ List

To configure resourceFiles to stream from a FASTQ file list file, use the following command. The FASTQ files in the following command are located on Blob storage account.

LIST_URL=
az storage blob generate-sas \
--name <Blob path to FASTQ list> \
--account-name <Storage account name> \
--account-key <Access key for storage account> \
--container-name <Blob container name> \
--expiry <Date and time the SAS token expires> \
--permissions r \
--https \
--full-uri \
--output tsv

The task.json file is structured as follows.

"resourceFiles": [{
"filePath": "dragen.tar",
"httpUrl": "$GENOME_URL"
}, {
"filePath": "fastq_list.csv",
"httpUrl": "$LIST_URL"
}]

Use the following command as the Batch task command. If using the following command, you do not need to specify resourceFiles in task.json.

/bin/bash -c \
"echo DefaultEndpointsProtocol=https >> ~/.azure-credentials; \
echo AccountName=<Blob storage account name> >> ~/.azure-credentials; \
echo AccountKey=<Access key for storage account> >> ~/.azure-credentials; \
echo EndpointSuffix=core.windows.net >> ~/.azure-credentials; \
mkdir dragen output; \
tar xvf dragen.tar -C dragen; \
/opt/edico/bin/dragen --partial-reconfig HMM --ignore-version-check true; \
/opt/edico/bin/dragen -f -r dragen \
--fastq-list fastq_list.csv \
--fastq-list-sample-id <RGSM> \
--enable-bam-indexing true \
--enable-map-align-output true \
--enable-sort true \
--output-file-prefix dragen-batch \
--enable-map-align true \
--output-format BAM \
--output-directory output \
--enable-variant-caller true \
--lic-server <LICENSE>"

Create the Batch Task

After you have set up the task.json file, you can use this file and Batch job ID to create the Azure Batch task with the following command.

For more information on creating Azure Batch tasks, see the Azure Batch documentation available on the Microsoft site.

az batch task create \
--job-id <ID of the Batch job> \
--json-file task.json