ORA Compression

DRAGEN ORA Compression is a fully lossless compression, that compresses *.fastq and *.fastq.gz files into *.fastq.ora files. DRAGEN ORA supports FASTQ generated by Illumina sequencing systems. When using the ORA format, the md5 checksum of the FASTQ content is preserved after a compression and decompression cycle to ensure a lossless compression.

DRAGEN ORA Compression requires a separate license. Decompression and ingestion of *.fastq.ora files into the DRAGEN map/align does not require a license. If your DRAGEN server is connected to a network, DRAGEN ORA compression can be used after installing DRAGEN v3.8 or later. If your DRAGEN server is offline, contact Illumina Customer Service.

For human data generated by the NovaSeq 6000, NextSeq 1000, or NextSeq 2000 sequencing systems, the compression ratio is expected to be up to 6x compared to the *.fastq.gz. The compressed file uses the *.fastq.ora extension.

Input of DRAGEN ORA Compression is *.fastq or *.fastq.gz. Input can be a single file or a list of files. A list of files can be specified on the command line, or from a *.fastq-list.csv generated by the BCL Convert BaseSpace Sequence Hub App or DRAGEN BCL convert. Input located in local storage, AWS S3 or Azure Blob storage is supported.

*.fastq.ora files are decompressed into *.fastq.gz.

Note: *.fastq.ora can be generated starting from BCL. To convert BCL into *.fastq.ora, specific commands need to be used. Follow the DRAGEN ORA compression from BCL instructions.

ORA Reference

To compress or decompress ORA files, you must provide the ORA reference files and specify an ORA reference directory.

Several references to compress data from different species and from different type of human data are supported. Refer to the list of supported references below.

You can download ORA reference files from the DRAGEN Software Support Site page. To ensure proper management of the reference files, do not change any of the file names of the downloaded archive.

To specify an ORA reference directory, do as follows.

  1. Download the oradata-2.tar.gz (or archive relevant to your studied model) from the DRAGEN Software Support Site.

  2. Move the file to the location you would like to contain the reference directory in, and then enter the following to extract the contents. tar -xzvf oradata-2.tar.gz

  3. Set the --ora-reference command line option to the extracted /oradata folder path.

The oradata folder should follow the following structures:

When only one reference is handled:

oradata
  ├── lena_index_V2
  └── refbin

--ora-reference should still point to the parent oradata folder.

When one ore more references are handled:

oradata
├── homo_sapiens_bisulfite
│   ├── lena_index_V2
│   └── refbin
├── gallus_gallus
│   ├── lena_index_V2
│   └── refbin
└── mus_musculus
    ├── lena_index_V2
    └── refbin

You can select at compression which reference species to use with option --ora-compression-species <species_scientific_name>. If unspecified, Homo sapiens reference will be used by default. Using a reference species that does not match the organism sequenced in your FASTQ file will still produce valid ORA compressed file, albeit with lower compression ratio. If the oradata folder pointed by --ora-reference does not contain the requested species, DRAGEN will stop with error. At decompression, detection of the species used to compress the ORA file is automatic. DRAGEN will look for the appropriate species in the oradata folder pointed by --ora-reference. If it is missing, DRAGEN will stop with an error message indicating the name of the missing species. In that case download it from the DRAGEN Software Support Site page.

Command Line Options

The following example command contains the required DRAGEN ORA compression options to compress regular human data:

dragen --enable-map-align false --ora-input <FILE> --enable-ora true --ora-reference <...> --output-directory <...>

or

dragen --enable-map-align false --fastq-list <FILE .csv> --enable-ora true --ora-reference <...> --output-directory <...>

The following example command contains the required DRAGEN ORA decompression options (for human and non-human data):

dragen --enable-map-align false --ora-input <FILE> --enable-ora true --ora-decompress true --ora-reference <...> --output-directory <...>

The following examples command contains the required options to compress FASTQs of a fastq-list.csv file containing multiple samples (regular human data):

When all samples must be compressed: dragen --enable-map-align false --fastq-list <FILE .csv> --enable-ora true --fastq-list-all-samples true --ora-reference <...> --output-directory <...>

When only specific samples must be compressed: dragen --enable-map-align false --fastq-list <FILE .csv> --enable-ora true --fastq-list-sample-id <sample> --ora-reference <...> --output-directory <...>

The following examples command contains the required options to achieve an interleaved compression of paired-read files from a fastq-list.csv file (regular human data) :

dragen --enable-map-align false --fastq-list <FILE .csv> --enable-ora true --ora-interleaved-compression true --ora-reference <...> --output-directory <...>

The following example command contains the required DRAGEN ORA compression options to compress non-human or specific human data, chicken data in this case:

dragen --enable-map-align false --ora-input <FILE> --enable-ora true --ora-compression-species <gallus_gallus> --ora-reference <...> --output-directory <...>

The following example command prints the file information summary of an ORA compressed file. Compression or decompression is not performed.

dragen --enable-map-align false --ora-input <FILE> --enable-ora=true --ora-print-file-info

The following example command compares FASTQ file checksum and decompressed FASTQ.ORA file checksum and outputs "ORA integrity check successful" if both checksums are equal or "integrity check failed" if checksums are not equal.

dragen --enable-map-align false --ora-input <FILE> --enable-ora=true --ora-reference <...> --ora-check-file-integrity=true

The following example command contains the required DRAGEN ORA compression options to print the list of available references:

dragen --enable-map-align false --enable-ora true --ora-list-species true

The following are the command line options for running DRAGEN ORA Compression and Decompression.

Use the --output-directory option to specify the directory to store output compressed/decompressed files.

Interleaved Compression

There are two methods to achieve a paired compression aka interleaved compression:

  • when using --ora-input and --ora-input2. The nth file of the --ora-input list is compressed together with the nth file of the --ora-input2

  • when using --fastq-listand --ora-interleaved-compression set to true. The paired-read files from the nth line of fast-list.csv are compressed together

Both files are interleaved within a single ORA output file with file name containing -interleaved. Using these options to compress paired files together improves compression by up to 10%. If decompressing an ORA file that contains paired data, the file is automatically decompressed to two separate files. To map an ORA file that contains paired interleaved data with the DRAGEN mapper, use the --interleaved option.

How to use ORA input files with DRAGEN Map/Align

DRAGEN can directly process ORA files. The same options as the other FASTQ input file types can be used. To use the ORA file, replace the FASTQ file name with the ORA file name and specify the ORA reference directory using --ora-reference.

The following command represents paired-end in two matched ORA FASTQ files (-1 and -2 options).

dragen -r <REF_DIR> -1 <fastq.ora1> -2 <fastq.ora2> \
--ora-reference <ORADATA_DIR> \
--output-directory <OUT_DIR> --output-file-prefix <OUT_PREFIX> \
--RGID <RGID> --RGSM <RGSM>

List of supported references

Below is a list of supported references. This list may not be exhaustive, the most up-to-date list of supported references can be found on the DRAGEN Software Support Site page.

Either the whole database or specific references can be dowloaded.

Last updated