Use the following information to run the DRAGEN virtual machine (VM) on Microsoft Azure Batch. For information on using DRAGEN, see the see the DRAGEN User Guide Section. For information on using Azure, see the Azure documentation available on the Microsoft site.
Navigate to the Microsoft Azure portal, and then sign in.
Select Marketplace.
Select View Private Offers, and then select DRAGEN on Azure.
Select Create. Starting from a preset configuration option is not recommended for DRAGEN.
Select a subscription and resource group from the drop-down menus, or select Create New.
Enter a name for the virtual machine.
Select a region that is compatible with the NP-series. See the Azure documentation available on the Microsoft site for more information.
Select DRAGEN and the current version as the image.
Select a storage size from the Size drop-down list. Only NP10 and NP20 sizes are compatible.
Configure any additional VM settings. For your disk type, DRAGEN recommends using Premium SSD for optimal performance. For information, see the Azure documentation available on the Microsoft site.
When finished, select Review + Create
To launch the VM, select Create.
After deployment completes, select Go to Resource. After your VM deploys, you can connect to the DRAGEN VM via the Azure Cloud Shell or another client of your choice. For more information on using a VM, see the Azure documentation available on the Microsoft site. For more information on DRAGEN analysis and command line options,
You can also run DRAGEN on Azure Batch using an Azure Resource Manager (ARM) template available on the DRAGEN Multi-Cloud support site.
The ARM template only includes the parameters required to run DRAGEN on Azure Batch. See the Microsoft Azure documentation available on the Microsoft site for information on configuring additional parameters.
Running DRAGEN using an ARM template enables the following advanced options.
Incorporating DRAGEN into an existing infrastructure.
Automating deployments with CI/CD pipelines.
Customizing the DRAGEN deployment.
The ARM template available on the DRAGEN Multi-Cloud support site creates a storage account and container. To use an existing Azure Blob storage account, specify the following input parameters in the ARM template.
storageNewOrExisting: existing
storageAccountName: <name of your existing storage account>
Use the following instructions to run DRAGEN on Azure Batch using the ARM template available on the DRAGEN Multi-Cloud support site.
Download the ARM template available on the DRAGEN Multi-Cloud support site.
Enter the following commands.
You can enter additional command line options to further customize the run, including maximum Batch job and task run time. See the Azure CLI documentation available on the Microsoft site for more information.
After creating and authenticating your Azure Batch account, use the following instructions to run DRAGEN with the Azure Batch CLI.
To run a DRAGEN Batch task, create a task.json file. The task.json file contains information on the Batch task, resource files, and output files. See Create the JSON file for information. You can then use the JSON file in the create Batch task command.
For more information on creating Azure Batch accounts and using the Azure Batch CLI, see the Azure Batch documentation available on the Microsoft site.
To set up the task.json to use in the Batch task create command, use the following structure.
Use the following instructions to configure the Batch task. For more information using the Azure Batch CLI, see the Azure Batch documentation available on the Microsoft site.
Create the batch job using the following command.
Enter the following command line to create the batch task. To use files located in a private Azure Blob storage account, see the Azure Blob CLI documentation available on the Microsoft site.
The following is an example Batch task configuration.
To add resource files to the Batch node, use the resourceFiles configuration in the task.json. The following example specifies the genome and FASTQ files. To use files located in a private Azure Blob storage account, see the Azure Blob CLI documentation available on the Microsoft site.
To specify the location to write output files, use the outputFiles configuration in the task.json. The following command-line example places output logs and DRAGEN files in the specified storage container. To generate a storage container URL, use a SAS token. For more information on accessing a Blob storage container using a SAS token, see the Azure Blob CLI documentation available on the Microsoft site.
The following is an example task.json file.
DRAGEN can stream input FASTQ and BAM files from private Azure Blob containers. The genome file must be located locally on the node. DRAGEN does not support streaming from public Blob containers.
Stream From Azure Blob Storage
Use the following command as the Batch task command. If using the following command, you do not need to specify resourceFiles in task.json.
Stream From FASTQ List
You can use a FASTQ list file to reference and stream FASTQ files. The FASTQ list file must be local to the node. The FASTQ files referenced in the FASTQ list can be URLs to files on a Blob storage account.
To configure resourceFiles to stream from a FASTQ file list file, use the following command. The FASTQ files in the following command are located on Blob storage account.
The task.json file is structured as follows.
Use the following command as the Batch task command. If using the following command, you do not need to specify resourceFiles in task.json.
After you have set up the task.json file, you can use this file and Batch job ID to create the Azure Batch task with the following command.
For more information on creating Azure Batch tasks, see the Azure Batch documentation available on the Microsoft site.
You can run DRAGEN analysis on Microsoft Azure Batch. For information on using Azure, see the Azure documentation available on the Microsoft site.
If you are not using an Azure Batch account, see DRAGEN on Microsoft Azure for information on using DRAGEN without Azure Batch.
To use DRAGEN on Azure, the following are required:
DRAGEN Multi-Cloud license. For more information, contact Illumina Technical Support.
Azure cloud subscription.
Quota for NP-series virtual machines (VM).
Azure command-line interface (CLI). For instructions on installing the Azure CLI, see the Azure documentation available on the Microsoft site.
Genomic data uploaded to your Azure Blob storage account.
The Azure directory principal must be assigned the Contributor role for the Azure subscription. If you would like to restrict access, you can run deployment pipelines as a managed service principal with the Contributor role.
DRAGEN on Azure is available in the following regions with field programmable gate array (FPGA)-enabled, standard NP-series virtual machines.
West US 2
East US
South Central US
West Europe (Amsterdam)
Southeast Asia (Singapore)
DRAGEN requires FPGA hardware. You must request access to the FPGA-enabled, NP-series VMs.
The vCPU requirements for NP-series SKUs are in increments of 10. When requesting an updated quota, DRAGEN recommends requesting vCPUs in batches of 10. The instructions for running DRAGEN on Azure require a minimum increase of 10 vCPU quotas for NP-series machines.
To make sure you have sufficient quota for NP-series VMs or to increase your quota, navigate to your Microsoft Azure portal, and then do as follows:
Select Subscriptions, then choose your subscription.
Under Settings, select Usage + quotas.
Enter NP into the search bar to filter for the NP-series. The quota list displays Standard NPS Family vCPUs.
If you do not see any results, select Request Increase. For more information on requesting quota for NP-series VMs, see the Azure documentation available on the Microsoft site.
By default, Azure Batch allocates pools using the Batch service mode. In Batch service mode, compute nodes are separated into different quotas. To run DRAGEN in Batch service mode, you need to request additional quota for your Azure Batch account. For information on the current default quota for Batch accounts and on increasing your Azure Batch account quota, see the Azure Batch documentation available on the Microsoft Azure site.
If using user subscription mode for your Batch account, you must add the Azure Batch service as the Contributor role. For more information, see the Microsoft Azure Batch documentation available on the Microsoft site.