Tools and Utilities
Monitoring System Health
When DRAGEN runs, a daemon dragend is started that communcates with the FPGA card. This daemon will monitor FPGA temperature while DRAGEN is running and abort DRAGEN when the temperature exceeds a configured threshold.
To display the current temperature of the DRAGEN server FPGA, use the dragen_info -t command.
% dragen_info -t
FPGA Temperature: 42C (Max Temp: 49C, Min Temp: 39C)Logging
All hardware events are logged to /var/log/messages and /var/log/dragen_mond.log. The following shows an example in /var/log/messages of a temperature alarm:
Jul 16 12:02:34 komodo dragen_mond[26956]: WARNING: FPGA software over temperature alarm has been triggered -- temp threshold: 85 (Chip status: 0x80000001)
Jul 16 12:02:34 komodo dragen_mond[26956]: Current FPGA temp: 86, Max temp: 88, Min temp: 48
Jul 16 12:02:34 komodo dragen_mond[26956]: All dragen processes will be stopped until alarm clears
Jul 16 12:02:34 komodo dragen_mond[26956]: Terminating dragen in process 1510 with SIGUSR2 signalBy default, temperature is logged to /var/log/dragen_mond.log every hour:
Aug 01 09:16:50 Setting FPGA hardware max temperature threshold to 100
Aug 01 09:16:50 Setting FPGA software max temperature threshold to 85
Aug 01 09:16:50 Setting FPGA software min temperature threshold to 75
Aug 01 09:16:50 FPGA temperatures will be logged every 3600 seconds
Aug 01 09:16:50 Current FPGA temperature is 52 (Max temp = 52, Min temp = 52)
Aug 01 10:16:50 Current FPGA temperature is 53 (Max temp = 56, Min temp = 49)
Aug 01 11:16:50 Current FPGA temperature is 54 (Max temp = 56, Min temp = 49)If DRAGEN is executing when a thermal alarm is detected, the following is displayed in the terminal window of the DRAGEN process:
**********************************************************
** Received external signal -- aborting dragen. **
** An issue has been detected with the dragen card. **
** Check /var/log/messages for details. **
** **
** It may take up to a minute to complete shutdown. **
**********************************************************If you see this message, stop running the DRAGEN software. Do the following to alleviate the overheating condition on the card:
Be sure that there is ample air flow over the card. Consider moving the card to a slot where there is more air flow, adding another fan or increasing the fan speed.
Give the card more space in the box. If there are available PCIe slots, move the card so that it has empty slots on either side.
Contact Illumina Technical Support if you are having trouble resolving the thermal alarm on your system.
Hardware Alarms
The following table lists the hardware events logged by the monitor when an alarm is triggered:
0
Software overheating
Terminate usage until DRAGEN server FPGA cools to software minimum temperature.
1
Hardware overheating
Fatal. Aborts dragen software; system reboot required
2
Board SPD overheating
Logged as nonfatal
3
SODIMM overheating
Logged as nonfatal
4
Power 0
Fatal. Aborts dragen software; system reboot required
5
Power 1
Fatal. Aborts dragen software; system reboot required
6
DRAGEN server FPGA power
Logged as nonfatal
7
Fan 0
Logged as nonfatal
8
Fan 1
Logged as nonfatal
9
SE5338
Fatal. Aborts dragen software; system reboot required
10--30
Undefined (Reserved)
Fatal. Aborts dragen software; system reboot required
Fatal alarms prevent the DRAGEN host software from running and require a system reboot. When a software overheating alarm is triggered, the monitor looks for and aborts any running DRAGEN processes. The monitor continues to abort any new DRAGEN processes until the temperature decreases to the minimum threshold and the hardware clears the chip status alarm. When the software overheating alarm clears, DRAGEN jobs can resume executing.
Contact Illumina Technical Support with details from the log files if any of these alarms are triggered on your system.
Hardware-Accelerated Compression and Decompression
Gzip compression is ubiquitous in bioinformatics. FASTQ files are often gzipped, and the BAM format itself is a specialized version of gzip. For that reason, DRAGEN provides hardware support for accelerating compression and decompression of gzipped data. If your input files are gzipped, DRAGEN detects that and decompresses the files automatically. If your output is BAM files, then the files are automatically compressed.
DRAGEN provides standalone command-line utilities to enable you to compress or decompress arbitrary files. These utilities are analogous to the Linux gzip and gunzip commands, but are named dzip and dunzip (dragen zip and dragen unzip). Both utilities are able to accept as input a single file, and produce a single output file with the .gz file extension removed or added, as appropriate. For example:
dzip file1 \# produces output file file1.gz\
dunzip file2.gz \# produces output file file2Currently, dzip and dunzip have the following limitations and differences from gzip/gunzip:
Each invocation of these tools can handle only a single file. Additional file names (including those produced by a wildcard * character) are ignored.
They cannot be run at the same time as the DRAGEN host software.
They do not support the command line options found in gzip and gunzip (eg, --recursive, --fast, ‑‑best, ‑‑stdout).
Licensing and Usage
For a DRAGEN server, there are two tools provided during the install process to assist with licensing of the system.
dragen_lic - A command line tool which can be utilized to perform licensing actions by a user.
dragen_licd - A daemon running in the background which communicates once a day with an Illumina License Server to perform automated licensing actions.
Usage reporting is a key component of the DRAGEN Licensing infrastructure. When a license is installed, any unreported usage data will automatically be uploaded to Illumina's License server. Note that Communication to the Illumina server is secured by encryption over an HTTP connection. Usage data entails the following information for each individual run
run date
run duration
licensing quota consumed (number of bases) in that run
run status
software version used for the run.
dragen_lic command line tool
Used to perform licensing actions by a user via the command line. For versions lower than 4.3, this tool can be found at /opt/edico/bin/dragen_lic. For 4.3 and higher, it can be found at /usr/bin/dragen_lic
Common actions would be to view licensing information (expiration date and remaining quota) and installing licenses. Examples of these actions are shown below.
Viewing License Information
There are three options for viewing licesing information. Examples for each one are below.
Default Output (i.e. no additional arguments). This is the legacy method for retreiving license information, and prior to DRAGEN v4.3 this was the only method available. In general it is recommended to instead use one of the following two options.
Basic Output (i.e. using the -b flag). This is the recommended method to view license information by a human user as the output is simplified to be more readable.
JSON Output (i.e. using the -j flag). This is the recommended method to view license information by a machine user as the output is already in a machine readable format.
$ dragen_lic
LICENSE_MSG| ---- Board (SN: <serial number>) ----
LICENSE_MSG| License Genome : used 68.0/100250 Gbases since 2023-Dec-05 (67961825900 bases, 0.1%)
LICENSE_MSG| issued=2023-Dec-04, start=2023-Dec-01, expiry=2023-Dec-19, period=15 months
LICENSE_MSG| License JointGenotype : used 0.0 Grecords since around 2023-Dec-04 (0 records, unlimited)
LICENSE_MSG| issued=2023-Dec-04, start=2023-Dec-01, expiry=2023-Dec-19, period=15 months
LICENSE_MSG| License Transcriptome : used 0.0 Gbases since around 2023-Dec-04 (0 bases, unlimited)
LICENSE_MSG| issued=2023-Dec-04, start=2023-Dec-01, expiry=2023-Dec-19, period=15 months
LICENSE_MSG| License GATK-accelerated: used 0.0 Gbases since around 2023-Nov-06 (0 bases, unlimited)
LICENSE_MSG| issued=2023-Nov-06, start=2021-Oct-13, expiry=2023-Dec-13, period=1 month
LICENSE_MSG| License Somatic : used 0.0 Gbases since 2023-Dec-04 (0 bases, unlimited)
LICENSE_MSG| issued=2023-Dec-04, start=2023-Dec-01, expiry=2023-Dec-19, period=15 months
LICENSE_MSG| License CNV : used 0.0 Grecords since around 2023-Dec-04 (0 records, unlimited)
LICENSE_MSG| issued=2023-Dec-04, start=2023-Dec-01, expiry=2023-Dec-19, period=15 months
LICENSE_MSG| License TSO500 : used 0.0 Gbases since around 2023-Nov-06 (0 bases, unlimited)
LICENSE_MSG| issued=2023-Nov-06, start=2021-Oct-13, expiry=2023-Dec-13, period=1 month
LICENSE_MSG| License TSO500Solid : used 24.0 Gbases since around 2023-Nov-06 (24000000000 bases, unlimited)
LICENSE_MSG| issued=2023-Nov-06, start=2021-Oct-13, expiry=2023-Dec-13, period=1 month
LICENSE_MSG| License Compression : used 0.0/100250 Gbases since 2023-Dec-04 (0 bases, 0.0%)
LICENSE_MSG| issued=2023-Dec-04, start=2023-Dec-01, expiry=2023-Dec-19, period=15 months
LICENSE_MSG| License TSO500_HRD : used 0.0 Gbases since around 2023-Nov-06 (0 bases, unlimited)
LICENSE_MSG| issued=2023-Nov-06, start=2021-Oct-13, expiry=2023-Dec-13, period=1 month
LICENSE_MSG| License TSO500Combined : used 23518260.0 Gbases since around 2022-Jul-15 (23518260000000000 bases, unlimited)
LICENSE_MSG| issued=2022-Jul-15, start=2022-Jul-13, expiry=2050-Jan-01, period=12 months
LICENSE_MSG| -- License dongle
LICENSE_MSG| STATUS : OK
LICENSE_MSG| DONGLE SN: <serial number>
LICENSE_MSG| RELEASE : <dongle release number>
LICENSE_MSG| CHIPID : *<chip ID>
LICENSE_MSG| DNA: active, accelerators=DNA
LICENSE_MSG| issue=2023-Dec-04, start=2021-Oct-13, expiry=2050-Jan-01
LICENSE_MSG| RNA: active, accelerators=RNA
LICENSE_MSG| issue=2023-Dec-04, start=2021-Oct-13, expiry=2050-Jan-01
LICENSE_MSG| GZIP: active, accelerators=GZIP
LICENSE_MSG| issue=2023-Dec-04, start=2021-Oct-13, expiry=2050-Jan-01
LICENSE_MSG| GUNZ: active, accelerators=GUNZ
LICENSE_MSG| issue=2023-Dec-04, start=2021-Oct-13, expiry=2050-Jan-01
LICENSE_MSG| HMM: active, accelerators=HMM
LICENSE_MSG| issue=2023-Dec-04, start=2021-Oct-13, expiry=2050-Jan-01
LICENSE_MSG| SMW: active, accelerators=SMW
LICENSE_MSG| issue=2023-Dec-04, start=2021-Oct-13, expiry=2050-Jan-01
LICENSE_MSG| RANS: active, accelerators=RANS
LICENSE_MSG| issue=2023-Dec-04, start=2021-Oct-13, expiry=2050-Jan-01
LICENSE_MSG| GRAPH: active, accelerators=GRAPH
LICENSE_MSG| issue=2023-Dec-04, start=2021-Oct-13, expiry=2050-Jan-01
LICENSE_MSG| DeepRTA: active, accelerators=DeepRTA
LICENSE_MSG| issue=2023-Dec-04, start=2021-Oct-13, expiry=2050-Jan-01$ dragen_lic -b
---- Board (SN: <serial number>) ----
DRAGEN Core:
Status: Active
Used: 68.0 Gbases since 2023-Dec-05
Quota: 100250 Gbases
Expiry: 2024-Feb-13
Includes; Genome, JointGenotype, CNV, Somatic, Transcriptome License(s)
Compression:
Status: Active - !!! EXPIRING IN LESS THAN 30 DAYS !!!
Used: 0 Gbases since 2024-Jan-12
Quota: Unlimited
Expiry: 2024-Feb-13
TSO500Combined:
Status: Active
Used: 23518284 Gbases since around 2022-Jul-15
Quota: Unlimited
Expiry: 2050-Jan-01
TSO500_HRD:
Status: Active - !!! EXPIRING IN LESS THAN 30 DAYS !!!
Used: 0 Gbases since around 2024-Jan-07
Quota: Unlimited
Expiry: 2024-Feb-13
Generated on 2024-01-16 18:54$ dragen_lic -j
{"dragen_boards":[{"board_sn":"<serial_number>","installed_licenses":{"Compression":{"expiry":"2025-Mar-19","quota_limit":100250,"remaining":95027,"start":"2023-Dec-01","status":"Active","units":"Gbases","used":5223},"TSO500Combined":{"expiry":"2050-Jan-01","quota_limit":-1,"start":"2022-Jul-13","status":"Active","units":"Gbases","used":23518284},"TSO500_HRD":{"expiry":"2024-Feb-13","quota_limit":-1,"start":"2021-Oct-13","status":"Active","units":"Gbases","used":0},"dragen_core":{"expiry":"2024-Feb-13","includes":{"CNV":{"expiration":"2025-Mar-19","status":"Active"},"Genome":{"expiration":"2024-Feb-13","status":"Active"},"JointGenotype":{"expiration":"2025-Mar-19","status":"Active"},"Somatic":{"expiration":"2025-Mar-19","status":"Active"},"Transcriptome":{"expiration":"2025-Mar-19","status":"Active"}},"quota_limit":-1,"start":"2023-Oct-14","status":"Active","units":"Gbases","used":79708}}}],"generated":1705098370}Basic mode (-b) will not show a license if it has been expired by more then 90 days.
Basic mode (-b) and JSON mode (-j) collapse the "DRAGEN Core" licenses into a single entry, as this is effectively how these licenses and bundled and sold.
Installing Licenses
To install licenses you must be running as a root user.
There are two options for installing licenses manually. Examples for each one are below.
Manual License File install (i.e. using the '-i <path to file>' argument). This can be used to install a license binary file (ending with the .bin extension).
Automatic License File install (i.e. using the '-i auto' argument). For DRAGEN servers which are connected to the internet, this will instruct the server to reach out to Illumina's License Server and download and automatically install the latest license file provided by Illumina. This is recommended method for installing a license.
sudo /usr/bin/dragen_lic -i /tmp/new_license.binsudo /usr/bin/dragen_lic -i autodragen_licd background daemon
For DRAGEN servers which are connected to the internet, this background process self-activates daily to automatically retrieve and install the latest license files associated with this server from Illumina's License Server. As part of this process, any unreported usage data is also uploaded.
Last updated
Was this helpful?