> For the complete documentation index, see [llms.txt](https://help.connected.illumina.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://help.connected.illumina.com/dragen/dragen-v4.6/reference/software-mode.md). # DRAGEN Software Mode ## Overview DRAGEN Software Mode runs DRAGEN secondary analysis on general-purpose CPU servers. It does not require a DRAGEN FPGA card, so you can run DRAGEN on an existing high performance computing (HPC) cluster, on a commodity Linux server, or on a standard cloud instance. Mapping and alignment, variant calling, and zip and unzip of FASTQ and BAM files run entirely in software. Processing time scales with thread count, which allows cost versus time tradeoffs. Use Software Mode where FPGA hardware is not available and where turnaround time is not critical, or when you run pipelines such as [Population Genotyping](/dragen/dragen-v4.6/product-guides/dragen-v4.6/dragen-dna-pipeline/iterative-gvcf-genotyper.md) that are designed for non-FPGA compute. To compare Software Mode with [FPGA Mode](/dragen/dragen-v4.6/reference/dragen-multi-cloud.md), and for all deployment choices, see [Deployment Options](/dragen/dragen-v4.6/overview/deployment-options.md). {% hint style="success" %} DRAGEN Software Mode produces exactly the same bit-exact output as DRAGEN FPGA Mode, deterministic across thread counts and deployments. {% endhint %} {% hint style="info" %} DRAGEN Software Mode is introduced with DRAGEN v4.6. This page applies to v4.6 and later. Earlier DRAGEN versions do not offer Software Mode as a supported deployment. {% endhint %} ## Requirements | Requirement | Value | | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | Operating system | Red Hat Enterprise Linux (RHEL) 8.x or 9.x, and RHEL-compatible Enterprise Linux distributions. | | Architecture | x86-64 | | CPU instructions | AVX2 (Advanced Vector Extensions 2) is mandatory. | | Threads | 16 or more available threads are required. 64 is recommended. | | Memory | 120 GB or more available system memory is required. 256 GB is recommended. | | Local storage | Fast local storage, ideally NVMe or SSD, on the compute node. Network-attached storage is supported but not recommended. | | File handles | `nofile` limit of 72000 or more. See [File handles and user processes](#file-handles-and-user-processes). | | User processes | `nproc` limit of 65535 or more. See [File handles and user processes](#file-handles-and-user-processes). | | Licensing | An [Illumina BioInsight Platform API Key](/dragen/dragen-v4.6/reference/licensing/api_key_licensing.md#running-dragen-on-your-own-infrastructure). | | Network | Outbound HTTPS access to `license.dragen.illumina.com` at runtime. | ### Sizing Recommendations DRAGEN is a natively multi-threaded application. The software automatically detects the number of system threads available, and aims to use the entire system's cores. The `--num-threads` option can be used to control how many processor cores DRAGEN will use (also referred to as vCPUs or Threads). DRAGEN can not be given RAM limits. Memory recommendations for sizing your compute: | Use Case | RAM Recommended (\*) | | ------------------------- | -------------------- | | WGS | 256GB | | WGS high coverage (>200x) | 384GB | | WGS low pass (<15x) | 128GB | | Exome | 128GB | | T/N Exome | 256GB | | Panel | 128GB | | 5-base WGS | 256GB | | 5-base Panel | 256GB | **\* Note:** Processing time scales with number of cores. * 64 vCPU is recommended as good balance for most workflows, including Germline WGS. * Panels and exomes can be run on fewer vCPUs, with minimal run time impact (32 vCPU). * Large Somatic Tumor-Normal WGS samples can benefit from more cores, if turn around time is important (96-128 vCPU). ### Recommended cloud instance types Use the latest available generation in your region. The following families are recommended for DRAGEN v4.6. | Cloud | Instance family | | ----- | ---------------------- | | AWS | m8a, m8i | | Azure | Dv7 series, Ev7 series | | GCP | c4d, n4d | | OCI | VM.Standard.E6.Flex | For provisioning, storage, and networking, refer to your cloud provider's documentation. ### AVX2 support Software Mode uses AVX2 vector instructions throughout the software mapper and the software variant calling engines. AVX2 has shipped in Intel processors since the Haswell architecture in 2013, and in AMD processors since 2015. DRAGEN checks for AVX2 before any other startup work, so a failure produces a bare message on stderr with no DRAGEN log file: ``` ERROR: Your processor does not support Intel/AMD AVX2 vector instructions required by DRAGEN. To see instruction set extensions supported by your processor run 'cat /proc/cpuinfo' and check field flags. ``` To confirm support before you install, check the CPU flags: ```bash grep -o avx2 /proc/cpuinfo | head -1 ``` ### Threads and memory DRAGEN checks available threads and memory prior to running. If the node has too few threads, DRAGEN reports the following, where `` is the value it detected: ``` Available threads is less than the minimum number of threads required 16 ``` If the node has too little memory, DRAGEN reports the following, where `` is the value it detected: ``` Available memory GB is less than the minimum system memory required 120GB ``` Both minimums are startup gates rather than hard hardware limits. `--min-threads INT` and `--min-memory INT` lower them, but Illumina does not recommend either option. {% hint style="warning" %} Lowering `--min-threads` or `--min-memory` below the defaults can cause long runtimes or out-of-memory failures. Move the run to a node that meets the requirements instead. {% endhint %} ### Network connectivity Every Software Mode run authenticates and meters usage against the Illumina DRAGEN License server, so each compute node requires outbound internet connectivity at runtime. Analysis fails if the node cannot reach the license server. Allow outbound HTTPS on port 443 to `license.dragen.illumina.com` from every node that runs DRAGEN, including all worker nodes in an HPC cluster. {% hint style="warning" %} Software Mode cannot run in an air-gapped or dark-site environment. There is no offline licensing option. {% endhint %} Confirm connectivity from each node before you submit jobs: ```bash curl https://license.dragen.illumina.com/healthcheck/version --header 'Content-Type: application/json' ``` ## Obtain and Install Software Mode is available as a self-extracting installer named `dragen-softwaremode-...bin` and as an RPM. Download either format from the **DRAGEN - Getting Started** app, which you can open from the Illumina BioInsight Platform home page or at [dragen.illumina.com](https://dragen.illumina.com). The steps below use the self-extracting installer. {% stepper %} {% step %} ### Confirm the node meets the requirements Verify the operating system, AVX2 support, recommended thread count, and available memory as described in [Requirements](#requirements). {% endstep %} {% step %} ### Download the installer In the DRAGEN - Getting Started app, download `dragen-softwaremode-...bin` for the RHEL major version and architecture of the target node. To fetch the installer directly on the node, copy the download link that the app provides. {% endstep %} {% step %} ### Run the installer By default the installer extracts to a directory beside itself: ```bash chmod +x dragen-softwaremode-...bin ./dragen-softwaremode-...bin ``` The extracted target directory is `$(pwd)/dragen-softwaremode-..`. {% endstep %} {% step %} ### Configure licensing Retrieve an Illumina BioInsight Platform API Key before your first run, and confirm the node can reach `license.dragen.illumina.com`. See [Licensing](#licensing) and [Network connectivity](#network-connectivity). {% endstep %} {% step %} ### Confirm the installation Run the DRAGEN binary from the extracted directory: ```bash /bin/dragen --version ``` {% endstep %} {% endstepper %} ## Enable Software Mode There are two ways to put a run into Software Mode. Either one is sufficient, and both reach the same execution mode. ### Dedicated package The `dragen-softwaremode` installer enables Software Mode for every run on that installation, so no command line option is required. Build the command line exactly as you would for an FPGA Mode run: ```bash dragen \ --api-key-file \ --ref-dir \ --output-directory \ --output-file-prefix \ -1 \ -2 \ --RGID \ --RGSM \ --enable-map-align true \ --enable-variant-caller true ``` ### The `--sw-mode` option `--sw-mode` requests Software Mode explicitly for a single run when not using the dedicated Software Mode package. ```bash dragen \ --sw-mode \ --api-key-file \ --ref-dir \ --output-directory \ --output-file-prefix \ -1 \ -2 \ --RGID \ --RGSM \ --enable-map-align true \ --enable-variant-caller true ``` | Option | Type | Description | | ----------------- | ---- | ---------------------------- | | `--sw-mode`, `-s` | FLAG | Run DRAGEN in Software Mode. | `--sw-mode` takes no value. Its presence on the command line, or in a configuration file as `sw-mode = true`, is what activates the mode. ## Licensing Software Mode is licensed with an Illumina BioInsight Platform API Key. Usage is metered and charged in BioInsight Credits (BICs). Because metering happens at runtime, each node needs network access to the license server. See [Network connectivity](#network-connectivity). For how to obtain a key and how to supply it to DRAGEN, see [BioInsight Platform Licensing](/dragen/dragen-v4.6/reference/licensing/api_key_licensing.md#running-dragen-on-your-own-infrastructure). ### Pipelines not supported DRAGEN Software Mode does not yet support the following: | Pipeline | | -------- | | TruPath | | RNA | | Spatial | | FastQC | ## Performance and Tuning ### Threads In Software Mode, DRAGEN defaults to all cores available on the node, which differs from FPGA Mode. This default is also the recommended setting. | Option | Type | Description | | --------------- | ---- | ------------------------------------------------------------------------------------------------------------- | | `--num-threads` | INT | Number of processor threads to use. Must be a positive number. (default=all available cores in Software Mode) | Set `--num-threads` below the core count only when you deliberately share a node, or when you want to reserve cores for other work. DRAGEN enables thread affinity whenever `--num-threads` is set. ### Local storage DRAGEN analysis is I/O intensive, and disk throughput can limit performance, affecting total runtime. * Use fast local storage, ideally NVMe or SSD on the compute node, for inputs, outputs, and temporary files. * Keep the reference directory on local storage. Every node reads it on every run. Network-attached storage is supported but not recommended. ### Cloud storage * Input streaming of FASTQ files from S3 or Azure Blob Storage is supported. * Output streaming of temporary files and output files to S3 or Azure Blob Storage is not supported. * Cloud instances must use attached storage such as EBS or FSx for temporary and output files. ### File handles and user processes Software Mode opens more file handles than an FPGA Mode run, because more of the pipeline runs as host threads. Raise the limits before you run. * File handles: allow `num_threads * (batch_size + reference + output file + 1 index file at a time)`, plus `stdin`, `stdout`, `stderr`, and the log file. * User processes: allow at least 65,000. The values DRAGEN needs are above the default per-user maximum in Linux, so `ulimit -n` and `ulimit -u` cannot raise them on their own. Create a limits configuration file instead: ```bash sudo tee /etc/security/limits.d/dragen.conf < ERROR: Your processor does not support Intel/AMD AVX2 vector instructions required by DRAGEN. The CPU does not support AVX2. DRAGEN performs this check before it initializes logging, so there is no DRAGEN log file for the failed run and the message appears only on stderr. Confirm CPU support with `grep -o avx2 /proc/cpuinfo`. If the command returns nothing, the node cannot run Software Mode. On a heterogeneous cluster, restrict submission to nodes with AVX2 using a scheduler constraint.
Available threads <threads> is less than the minimum number of threads required 16 The node exposes fewer than 16 threads, or the scheduler allocated fewer threads than the node has. Run on a node with more threads available.
Available memory <memory>GB is less than the minimum system memory required 120GB The node has less than 120 GB of system memory available. Run on a node with more memory available.
ERROR: <pipeline> pipeline is not supported by Software Only mode. The run requested a pipeline that is not available in Software Mode. Run that analysis on [DRAGEN FPGA Mode](/dragen/dragen-v4.6/reference/dragen-multi-cloud.md). See [Pipelines not supported](#pipelines-not-supported).
ERROR: Software mode (--sw-mode) can't be used with hardware ORA (--ora-use-hw=true) The run requested hardware-accelerated ORA compression or decompression. Omit `--ora-use-hw`, or set `--ora-use-hw=false`, to use the software ORA path.
ERROR: Software mode (--sw-mode) can't be used with hardware BCL (--bcl-use-hw=true, --bcl-conversion-only=true) The run requested hardware BCL conversion. This also occurs when `--bcl-conversion-only=true` is set and `--bcl-use-hw` is left at a value that implies hardware. Set `--bcl-use-hw=false` explicitly for BCL conversion in Software Mode.
An Illumina BioInsight Platform API Key must be provided. No API key was found. DRAGEN looks for credentials in the license configuration file, the `--api-key-file` path, the `DRAGEN_API_KEY_VALUE` environment variable, and the DRAGEN user and internal default configuration files. The full message lists both supported methods: ``` You can provide the API key to DRAGEN in one of two ways: 1) Environment variable: DRAGEN_API_KEY_VALUE= 2) File, referenced by file path: Pointed to by either --api-key-file or the DRAGEN_API_KEY_FILE environment variable. Note: This file shall contain nothing but a single line with the API key. Configure the API Key and rerun the command to proceed with software mode. ``` An API key file must contain a single line holding only the key. For setup instructions, see [BioInsight Platform Licensing](/dragen/dragen-v4.6/reference/licensing/api_key_licensing.md#set-up-api-key-licensing).
A run fails at licensing on some nodes but succeeds on others The nodes that fail cannot reach the Illumina DRAGEN License server. This is a common symptom on HPC clusters where only login nodes have outbound internet access, or where a subset of worker nodes sits behind a different firewall rule. Run the healthcheck from a failing node rather than from the login node: ```bash curl https://license.dragen.illumina.com/healthcheck/version --header 'Content-Type: application/json' ``` Allow outbound HTTPS to `license.dragen.illumina.com` from every worker node. See [Network connectivity](#network-connectivity).
For general DRAGEN troubleshooting, see [Troubleshooting](/dragen/dragen-v4.6/reference/troubleshooting.md). For help from Illumina, see [Support](/dragen/dragen-v4.6/reference/technical-assistance.md). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://help.connected.illumina.com/dragen/dragen-v4.6/reference/software-mode.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.