1 of 2

Launch DRAGEN Pipelines on CLI

Prerequisite: configure and authenticate the ICA command line interface (CLI).

Configure CLI and Identify Pipeline ID

Obtain a list of your projects with their associated IDs:

Use the ID of the project from the previous step to enter the project context:

Find the pipeline you want to start from the CLI by obtaining a list of pipelines associated with your project:

Find the ID associated with your pipeline of interest.

Identify Input File Parameters

To find the input files parameter, you can use a previously launched projectanalyses with the input command.

Find the previous analyses launched along with their associated IDs:

List the analyses inputs by using the ID found in the previous step:

This will return the Input File Codes, as well as the file names and data IDs of the associated data used to previously launch the pipeline

Identify Configuration Settings

You need to use the ICA API to access the configuration settings of a project analyses that ran successfully.

API-based Configuration Settings

Generate JWT Token from API Key or Basic login credentials
on how to get an API Key https://illumina.gitbook.io/ica/account-management/am-iam#api-keys
If your user has access to multiple domains, you will need to need to add a "?tenant=($domain)" to the request

Response to this request will provide a JWT token {"token":($token)}, use the value of the token in further requests
Using the API endpoint /api/projects/{projectID}/analyses/{analysisId}/configurations to find the configuration file listing out all of required and optional parameters

The response JSON to this API will have configuration items listed as

Nextflow XML file parameters via GUI (Prior to DRAGEN 4.3)

Click the previous GUI run, and select the pipeline that was run. On the pipeline page, select the XML Configuration Tab to view the configuration settings.

In the "steps" section of the XML file, you will find various steps labeled with

and subsequent labels of parameters with a similar structure

The code should be used to generate the later command line parameters e.g.

--parameters enable_map_align:true

Create Launch Command

CWL

Structure of the final command

icav2 projectpipelines start cwl $(pipelineID) --user-referenc Plus input options

Input Options - For CLI, the entire input can be broken down as individual command line arguments

To launch the same analysis as using the GUI, use the same file ID and parameters, if using new data you can use the CLI command icav2 projectdata list to find new file IDs to launch a new instance of the pipeline Required information in Input - Input Data and Parameters

Command Line Arguments

This option requires the use of --type input STRUCTURED along with --input and --parameters
The input parameter names such as Reference and Tumor_FASTQ_Files in the example below are from the pipeline definition where you can give the parameters a name. You can see which of these were used when the pipeline originally ran, in the section above. You can also look at the pipeline definitions for the input parameters, for example the code value of .

Successful Response

Unsuccessful: Pipeline ID not formatted correctly

Check that the pipeline ID is correct based on icav2 projectpipelines list

Unsuccessful: File ID not found

Check that the file ID is correct based on icav2 projectdata list

Unsuccessful: Parameter not found

Nextflow

When using nextflow to start runs, the input-type parameter is not used, but the --project-id is required

Structure of the file command icav2 projectpipelines start nextflow $(pipelineID) --user-reference Plus input options

The Response can be used to determine if the pipeline was submitted successfully.

Mount projectdata using CLI

icav2 allows project data to be mounted on a local system. This feature is currently available on Linux and Mac systems only. Although not supported, users have successfully used Windows Subsystem for Linux (WSL) on Windows to use icav2 projectdata mount command. Please refer to the WSL documentation for installing WSL.

Prerequisites

icav2 (>=2.3.0) and in the local system.
- For MAC refer to .

Mount projectdata

Identify the project id by running the following command:

Provide the project id under "ID" column above to the mount command to mount the project data for the project.

Check the content of the mount.

icav2 utilizes the FUSE driver to mount project data, providing both read and write capabilities. However, there are some limitations on the write capabilities that are enforced by the underlying AWS S3 storage. For more information, please refer to .

WARNING Do NOT use the CP -f command to copy or move data to a mounted location. This will result in data loss as data on the destination location will be deleted.

Unmount project data

You can unmount the project data using the 'unmount' command.

Launch DRAGEN Pipelines on CLI

Prerequisite: configure and authenticate the ICA command line interface (CLI).

Configure CLI and Identify Pipeline ID

Obtain a list of your projects with their associated IDs:

icav2 projects list

ID                                      NAME         OWNER
a5690b16-a739-4bd7-a62a-dc4dc5c5de6c    Project1     670fd8ea-2ddb-377d-bd8b-587e7781f2b5
ccb0667b-5949-489a-8902-692ef2f31827    Project2     f1aa8430-7058-4f6c-a726-b75ddf6252eb
No of items :  2

Use the ID of the project from the previous step to enter the project context:

icav2 projects enter a5690b16-a739-4bd7-a62a-dc4dc5c5de6c

Find the pipeline you want to start from the CLI by obtaining a list of pipelines associated with your project:

Find the ID associated with your pipeline of interest.

Identify Input File Parameters

To find the input files parameter, you can use a previously launched projectanalyses with the input command.

Find the previous analyses launched along with their associated IDs:

List the analyses inputs by using the ID found in the previous step:

This will return the Input File Codes, as well as the file names and data IDs of the associated data used to previously launch the pipeline

Identify Configuration Settings

You need to use the ICA API to access the configuration settings of a project analyses that ran successfully.

API-based Configuration Settings

Generate JWT Token from API Key or Basic login credentials
on how to get an API Key https://illumina.gitbook.io/ica/account-management/am-iam#api-keys
If your user has access to multiple domains, you will need to need to add a "?tenant=($domain)" to the request

Response to this request will provide a JWT token {"token":($token)}, use the value of the token in further requests
Using the API endpoint /api/projects/{projectID}/analyses/{analysisId}/configurations to find the configuration file listing out all of required and optional parameters

The response JSON to this API will have configuration items listed as

Nextflow XML file parameters via GUI (Prior to DRAGEN 4.3)

Click the previous GUI run, and select the pipeline that was run. On the pipeline page, select the XML Configuration Tab to view the configuration settings.

In the "steps" section of the XML file, you will find various steps labeled with

and subsequent labels of parameters with a similar structure

The code should be used to generate the later command line parameters e.g.

--parameters enable_map_align:true

Create Launch Command

CWL

Structure of the final command

icav2 projectpipelines start cwl $(pipelineID) --user-referenc Plus input options

Input Options - For CLI, the entire input can be broken down as individual command line arguments

Command Line Arguments

This option requires the use of --type input STRUCTURED along with --input and --parameters
The input parameter names such as Reference and Tumor_FASTQ_Files in the example below are from the pipeline definition where you can give the parameters a name. You can see which of these were used when the pipeline originally ran, in the section above. You can also look at the pipeline definitions for the input parameters, for example the code value of .

Successful Response

Unsuccessful: Pipeline ID not formatted correctly

Check that the pipeline ID is correct based on icav2 projectpipelines list

Unsuccessful: File ID not found

Check that the file ID is correct based on icav2 projectdata list

Unsuccessful: Parameter not found

Nextflow

When using nextflow to start runs, the input-type parameter is not used, but the --project-id is required

Structure of the file command icav2 projectpipelines start nextflow $(pipelineID) --user-reference Plus input options

The Response can be used to determine if the pipeline was submitted successfully.

analysisStorage.description           1.2 TB
analysisStorage.id                    6e1b6c8f-f913-48b2-9bd0-7fc13eda0fd0
analysisStorage.name                  Small
analysisStorage.ownerId               8ec463f6-1acb-341b-b321-043c39d8716a
analysisStorage.tenantId              f91bb1a0-c55f-4bce-8014-b2e60c0ec7d3
analysisStorage.tenantName            ica-cp-admin
analysisStorage.timeCreated           2021-11-05T10:28:20Z
analysisStorage.timeModified          2021-11-05T10:28:20Z
id                                    51abe34a-2506-4ab5-adef-22df621d95d5
ownerId                               47793c21-75a6-3aa8-8147-81b354d0af4d
pipeline.analysisStorage.description  1.2 TB
pipeline.analysisStorage.id           6e1b6c8f-f913-48b2-9bd0-7fc13eda0fd0
pipeline.analysisStorage.name         Small
pipeline.analysisStorage.ownerId      8ec463f6-1acb-341b-b321-043c39d8716a
pipeline.analysisStorage.tenantId     f91bb1a0-c55f-4bce-8014-b2e60c0ec7d3
pipeline.analysisStorage.tenantName   ica-cp-admin
pipeline.analysisStorage.timeCreated  2021-11-05T10:28:20Z
pipeline.analysisStorage.timeModified 2021-11-05T10:28:20Z
pipeline.code                         DRAGEN Somatic 3.9.5
pipeline.description                  The DRAGEN Somatic tool identifies somatic variants which can exist at low allele frequencies in the tumor sample. The pipeline can analyze tumor/normal pairs and tumor-only sequencing data. The normal sample, if present, is used to avoid calls at sites with germline variants or systematic sequencing artifacts. Unlike germline analysis, the somatic platform makes no ploidy assumptions about the tumor sample, allowing sensitive detection of low-frequency alleles.
pipeline.id                           fbd6f3c3-cb70-4b35-8f57-372dce2aaf98
pipeline.language                     CWL
pipeline.ownerId                      e9dd2ff5-c9ba-3293-857e-6546c5503d76
pipeline.tenantId                     55cb0a54-efab-4584-85da-dc6a0197d4c4
pipeline.tenantName                   ilmn-dragen
pipeline.timeCreated                  2021-11-23T22:55:49Z
pipeline.timeModified                 2021-12-09T16:42:14Z
reference                             kyle-test-somatic-9-DRAGEN Somatic 3_9_5-bc56d4b1-f90e-4039-b3a4-b11d29263e4e
status                                REQUESTED
summary
tenantId                              b5b750a6-49d4-49de-9f18-75f4f6a81112
tenantName                            ilmn-cci
timeCreated                           2022-03-16T22:48:31Z
timeModified                          2022-03-16T22:48:31Z
userReference                         kyle-test-somatic-9

analysisStorage.description           1.2 TB
analysisStorage.id                    6e1b6c8f-f913-48b2-9bd0-7fc13eda0fd0
analysisStorage.name                  Small
analysisStorage.ownerId               8ec463f6-1acb-341b-b321-043c39d8716a
analysisStorage.tenantId              f91bb1a0-c55f-4bce-8014-b2e60c0ec7d3
analysisStorage.tenantName            ica-cp-admin
analysisStorage.timeCreated           2021-11-05T10:28:20Z
analysisStorage.timeModified          2021-11-05T10:28:20Z
id                                    51abe34a-2506-4ab5-adef-22df621d95d5
ownerId                               47793c21-75a6-3aa8-8147-81b354d0af4d
pipeline.analysisStorage.description  1.2 TB
pipeline.analysisStorage.id           6e1b6c8f-f913-48b2-9bd0-7fc13eda0fd0
pipeline.analysisStorage.name         Small
pipeline.analysisStorage.ownerId      8ec463f6-1acb-341b-b321-043c39d8716a
pipeline.analysisStorage.tenantId     f91bb1a0-c55f-4bce-8014-b2e60c0ec7d3
pipeline.analysisStorage.tenantName   ica-cp-admin
pipeline.analysisStorage.timeCreated  2021-11-05T10:28:20Z
pipeline.analysisStorage.timeModified 2021-11-05T10:28:20Z
pipeline.code                         DRAGEN Somatic 3.9.5
pipeline.description                  The DRAGEN Somatic tool identifies somatic variants which can exist at low allele frequencies in the tumor sample. The pipeline can analyze tumor/normal pairs and tumor-only sequencing data. The normal sample, if present, is used to avoid calls at sites with germline variants or systematic sequencing artifacts. Unlike germline analysis, the somatic platform makes no ploidy assumptions about the tumor sample, allowing sensitive detection of low-frequency alleles.
pipeline.id                           fbd6f3c3-cb70-4b35-8f57-372dce2aaf98
pipeline.language                     CWL
pipeline.ownerId                      e9dd2ff5-c9ba-3293-857e-6546c5503d76
pipeline.tenantId                     55cb0a54-efab-4584-85da-dc6a0197d4c4
pipeline.tenantName                   ilmn-dragen
pipeline.timeCreated                  2021-11-23T22:55:49Z
pipeline.timeModified                 2021-12-09T16:42:14Z
reference                             kyle-test-somatic-9-DRAGEN Somatic 3_9_5-bc56d4b1-f90e-4039-b3a4-b11d29263e4e
status                                REQUESTED
summary
tenantId                              b5b750a6-49d4-49de-9f18-75f4f6a81112
tenantName                            ilmn-cci
timeCreated                           2022-03-16T22:48:31Z
timeModified                          2022-03-16T22:48:31Z
userReference                         kyle-test-somatic-9

Mount projectdata using CLI

Prerequisites

icav2 (>=2.3.0) and in the local system.
- For MAC refer to .

Mount projectdata

Identify the project id by running the following command:

Provide the project id under "ID" column above to the mount command to mount the project data for the project.

Check the content of the mount.

WARNING Do NOT use the CP -f command to copy or move data to a mounted location. This will result in data loss as data on the destination location will be deleted.

Unmount project data

You can unmount the project data using the 'unmount' command.

Launch DRAGEN Pipelines on CLI

hashtagConfigure CLI and Identify Pipeline ID

hashtagIdentify Input File Parameters

hashtagIdentify Configuration Settings

hashtagAPI-based Configuration Settings

hashtagCreate Launch Command

hashtagCWL

hashtagCommand Line Arguments

hashtagSuccessful Response

hashtagUnsuccessful: Pipeline ID not formatted correctly

hashtagUnsuccessful: File ID not found

hashtagUnsuccessful: Parameter not found

hashtagNextflow

Mount projectdata using CLI

hashtagMount projectdata using CLI

hashtagPrerequisites

hashtagMount projectdata

hashtagUnmount project data

Launch DRAGEN Pipelines on CLI

hashtagConfigure CLI and Identify Pipeline ID

hashtagIdentify Input File Parameters

hashtagIdentify Configuration Settings

hashtagAPI-based Configuration Settings

hashtagCreate Launch Command

hashtagCWL

hashtagCommand Line Arguments

hashtagSuccessful Response

hashtagUnsuccessful: Pipeline ID not formatted correctly

hashtagUnsuccessful: File ID not found

hashtagUnsuccessful: Parameter not found

hashtagNextflow

Mount projectdata using CLI

hashtagMount projectdata using CLI

hashtagPrerequisites

hashtagMount projectdata

hashtagUnmount project data

Configure CLI and Identify Pipeline ID

Identify Input File Parameters

Identify Configuration Settings

API-based Configuration Settings

Create Launch Command

CWL

Command Line Arguments

Successful Response

Unsuccessful: Pipeline ID not formatted correctly

Unsuccessful: File ID not found

Unsuccessful: Parameter not found

Nextflow

Mount projectdata using CLI

Prerequisites

Mount projectdata

Unmount project data

Configure CLI and Identify Pipeline ID

Identify Input File Parameters

Identify Configuration Settings

API-based Configuration Settings

Create Launch Command

CWL

Command Line Arguments

Successful Response

Unsuccessful: Pipeline ID not formatted correctly

Unsuccessful: File ID not found

Unsuccessful: Parameter not found

Nextflow

Mount projectdata using CLI

Prerequisites

Mount projectdata

Unmount project data