# Cloud Analysis Auto-launch

Cloud analysis auto-launch refers to the automated launch of secondary analysis by Illumina's cloud platforms after a sequencing run is completed. Coupled with the ability to automatically output sequencing data to a cloud location, the auto-launch functionality minimizes user touchpoints and decreases turnaround time.

Sequencing run management, including run planning, run monitoring, and viewing results, is performed via BaseSpace Sequence Hub (BaseSpace). The post-sequencing secondary analysis (ie, demultiplexing, DRAGEN) is powered by ICA pipelines. The analysis results are stored in an /ilmn-analyses/\<sequencer\_run\_output> subfolder in ICA. See the [Sequencer Auto-launch Analyses Compatibility](https://help.connected.illumina.com/analysis/sequencer-reference) table for available instruments and pipelines.

* **BaseSpace** is used for sequencing run planning and monitoring. See [BaseSpace Setup ](#basespace-setup)for more information.
* **ICA** is used to store sequencing run data and perform secondary analysis. See [Sequencer Run Data in ICA](#sequencer-run-data) for more information.

<figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-78be4dad01d829b672ef667eb49017aaeec698e3%2FanalysisAutolaunch1.png?alt=media" alt=""><figcaption></figcaption></figure>

1. **Run Planning** - performed in BaseSpace to configure a sequencing run including choosing instrument type, analysis, and sample settings
2. **Start Sequencing Run** - performed on the sequencing instrument to select the planned run and launch the run
3. **Data Upload and Run Monitoring**
   * Data Upload - the sequencing instrument uploads the sequencing run output data to ICA
   * Run Monitoring - sequencing run monitoring is available in BaseSpace
4. **Auto-launch Analysis** - the secondary analysis is automatically launched in ICA upon sequencing run completion
5. **Auto-launch Analysis Monitoring** - the secondary analysis status is monitored in BaseSpace
6. **Edit & Requeue SampleSheet to trigger #4 (optional)** - if an error occurs during secondary analysis, the analysis may be requeued (re-launched) from BaseSpace

Further details about each step above is described in the sections below.

## BaseSpace Setup

BaseSpace settings must be configured to send sequencing run data to ICA prior to sequencing. For workgroups, this must be configured by the workgroup administrator. Workgroup users have read-access to the toggle switch setting, but cannot modify it.

Navigate to the BaseSpace Settings view. In the "Settings" section, set the "ICA Run Storage" setting to ON.

{% hint style="info" %}
This setting is not necessary for NovaSeq X Series instrument type
{% endhint %}

{% hint style="info" %}
The visibility of the "ICA Run Storage" configuration setting is controlled by Illumina support. Contact Illumina Support to request the setting be made available for your domain.
{% endhint %}

<figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-d1e9fa1d3230c7aa4af17adddaafb58120b3597a%2Fimage%20(5)%20(1)%20(1)%20(1).png?alt=media" alt=""><figcaption></figcaption></figure>

## Sequencing Run Planning

Sequencing run planning in BaseSpace Sequence Hub is the first step in preparing a sequencing run with automated post-sequencing analysis.

A sequencing run can be started from the instrument using either:

1. Planned sequencing run setup in Basespace Run Planning (recommended). See [Plan the Sequencing Run](https://help.connected.illumina.com/cross-product-tutorials/novaseqx-bcl-autolaunch#plan-the-sequencing-run) for a guided example.
2. Sample Sheet imported during the on-instrument Run Setup. See [Cloud Analysis Auto-launch Settings](https://help.connected.illumina.com/run-set-up/overview/auto-launch-settings) for more information.

Refer to [Sequencer Autolaunch Analyses Compatibility](https://help.connected.illumina.com/analysis/sequencer-reference) for guidance on sequencer compatibility with different versions of analyses applications.

## Sequencing Run Monitoring in BaseSpace

Monitoring the status of an ongoing sequencing run is done through BaseSpace. See the [View Runs](https://help.basespace.illumina.com/data/view-data/view-runs) BaseSpace help documentation for more information.

1. Find the sequencing run in BaseSpace Sequence Hub by logging in, selecting the Runs tab, and selecting the appropriate sequencing run from the list.
2. When the sequencing run completes uploading, the run status in BaseSpace Sequence Hub will move to **Pending Analysis**. Analysis will automatically launch based on the information in the Sample Sheet.
   1. If there is an issue with the Sample Sheet, the status will move to **Needs Attention**. Use the Requeue feature to correct any issues and re-launch analysi&#x73;*.* See [Requeue Analysis](#requeue-analysis).
3. When the analysis launches, the run status will move to **Analyzing**. The associated analysis can be found on the Run Details tab under Latest Analysis, or in the Analysis list.
4. Click the analysis link to go to the **Analysis Details** page. Information about the analysis progress will be displayed in the Logs section.
   1. If the analysis aborts with an error, logs and any files that had been produced prior to the error will available in the Files tab.
5. When the analysis completes, any aggregate and/or sample reports can be viewed in the Reports tab. Logs and other output files can be viewed in the Files tab.

## View Sequencer Run Data and Analysis

{% tabs %}
{% tab title="View Files in ICA" %}

#### Sequencer Run Data in ICA <a href="#sequencer-run-data" id="sequencer-run-data"></a>

Sequencer run data is stored in ICA in an [externally-managed project](https://help.ica.illumina.com/home/h-projects#externally-managed-projects) created by BaseSpace. Operations in BaseSpace, including planning and monitoring instrument runs, are performed in a given user or workgroup *context*. To preserve permission boundaries between applications, an externally-managed project is created for each user/workgroup context per region. The ICA project will be named using the convention `BSSH {context name}`, where *{context-name}* is the user or workgroup name operating in BaseSpace. For workgroups, the project will be created on behalf of the workgroup owner. The project Team includes an entry for the workgroup to grant all workgroup users read-only access to the project. While Team members may be assigned the Contributor role in the externally-managed project, they will not have write access (ie, create/delete data, run pipelines) in the project.

In the example below, the project is created for a workgroup with name "Research" where the workgroup owner is "<bsadmin@illumina.com>".

<figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-244ec9964cbaf53989a836406a7006e612b3ed24%2Fimage%20(2)%20(1)%20(1)%20(1)%20(1).png?alt=media" alt=""><figcaption></figcaption></figure>

The project contains the following tags:

* Technical Tags: `bssh.project.user:{user/workgroup ID}`

For workgroups, users who are members of the workgroup will have access to the data in the externally-managed project.

All sequencing run data is stored in the `ilmn-runs` folder. Within that folder, the data for each sequencing run is stored in a sub-folder named based on the run name used to create the run during run planning (or as specified in the `RunName` row of the `[Header]` section in the sample sheet) appended with a unique identifier.

The screenshot below provides an example of a sequencer run output folder uploaded to ICA.

<figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-7eb248fd0493ae86f497d94d7f86f5e137c3bcb2%2Fimage%20(4)%20(1)%20(1)%20(1)%20(1).png?alt=media" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
Sequencer run data uploaded directly to ICA via CLI or Service Connector to a user-managed project will not be available in BaseSpace.
{% endhint %}

{% hint style="info" %}
Sequencer run data stored outside of ICA or BaseSpace may be uploaded using the BaseSpace CLI (command-line interface) to make it available in both BaseSpace and ICA.

Use the BaseSpace [Uploading a run](https://developer.basespace.illumina.com/docs/content/documentation/cli/cli-examples#Uploadingarunanamerun-uploada) instructions to upload a sequencer run output folder.
{% endhint %}

#### Automated Secondary Analysis in ICA

When the sequencing run data upload has completed, the post-sequencing secondary analysis described in the sample sheet will be automatically started.

In ICA, secondary analyses launched after a sequencing run completes is orchestrated by a parent process called a *Workflow Session*. The workflow session serves as a parent process orchestrating the secondary analysis, including launching secondary analysis pipelines and creating sample entities in ICA.

Upon completion of the sequencing run data upload, a workflow session record is created and visible in the ICA "Analyses" view. The workflow session record can be identified by the User Reference, which will have "ws" preceding the run name.

![](https://stratus-documentation-us-east-1-public.s3.amazonaws.com/images/newanalysisautolaunch1.png)

The workflow session details view includes a section **Orchestrated Analyses** to indicate analyses launched by the workflow session as part of the automated secondary analysis. These orchestrated analyses are driven by the information in the sample sheet, including the pipeline to launch, reference data, and sample-specific settings.

<figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-1ae16393c82b7ea6d5115c5d3b0b8a343d2e36ca%2Fnewsequencerguidedexample9.PNG?alt=media" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
Currently, only a limited set of Illumina-provided pipelines are compatible with workflow session automation.
{% endhint %}
{% endtab %}

{% tab title="View Files in BaseSpace" %}

#### Sequencer Run Data in BaseSpace <a href="#sequencer-run-data" id="sequencer-run-data"></a>

For users intending to use BaseSpace for FASTQ generation and DRAGEN Secondary Analysis, the data will be associated with several BaseSpace Projects.

**1**. **SampleSheet ProjectName**

When using ProjectName values in Run Planning, the FASTQ files will be written to those locations.

<figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-6cbcc3de9fd510bf511ff144c601c51b418d3d43%2FSampleSheetProjectName.png?alt=media" alt=""><figcaption></figcaption></figure>

In the SampleSheet.csv, this will be reflected in the \[Cloud\_Data] section.

<figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-30f23462416569d2d6b02099ffc1b716301ccfe3%2FSampleSheetCloudData.png?alt=media" alt=""><figcaption></figcaption></figure>

Once BCL Convert (and any associated downstream analysis) is completed, the FASTQ files will be associated with the ProjectName specified on BaseSpace. The project column is not required to be specified, and if left blank, the FASTQ Files will be written to a BaseSpace project following the nomenclature:

**2**. **bssh**\_\[**region**]\_\[**run ID**]

For example:

bssh\_use1-prod\_277283070

More information about regions can be found in the BaseSpace Help center article [Global Regions](https://help.basespace.illumina.com/manage-your-account/regions). The run ID corresponds to the numerical value observed in the web address bar when viewing the run.

<figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-4d695bcffaee1c50289a95c321d23786741bb31e%2FRunIDWebAddress.png?alt=media" alt=""><figcaption></figcaption></figure>

If any DRAGEN Secondary analysis is performed, this project will also contain the DRAGEN output.
{% endtab %}
{% endtabs %}

## Requeue Analysis

{% tabs %}
{% tab title="Requeue Analysis in ICA" %}

#### Manually Launch Analysis in ICA

There may be situations when an analysis needs to be launched manually in ICA, such as if the auto-launched analysis failed. Analyses can be launched manually in ICA with a few extra steps to prepare the input data.

For example, if the demultiplexing analysis fails due to a sample sheet mistake or a system error, the analysis can be launched manually in ICA by following these instructions:

1. Create a new or choose an existing user-managed project to link the data to and run the analysis in
2. Follow instructions to [Link Project Data](https://help.ica.illumina.com/project/p-data#link-project-data) to link the sequencing run output data to the user-managed project data

{% hint style="warning" %}
The "ilmn-runs" folder contains all sequencer run data managed by BaseSpace and should not be linked to a user-managed project. Only link individual sequencer run output folders within the ilmn-runs folder. See [Sequencer Run Data](#sequencer-run-data) for details on how to find the correct folder to link.
{% endhint %}

3. Link the Bundle containing the ICA pipeline to the user-managed project (ie, "DRAGEN Analysis 4.1.7 - Sequencer Integration Only") (See [Link Bundles](https://help.ica.illumina.com/project/p-details#link-bundles))
4. Fix the sample sheet by downloading the original sample sheet from the sequencing run output folder, modify, and re-upload the file to the user-managed project
5. Launch the pipeline (ie, BclConvert\_v4\_1\_7) with the corrected sample sheet as input

{% embed url="<https://www.youtube.com/watch?index=12&list=PLKRu7cmBQlaiQT6Giou9aSkZ4C0LMIGbc&v=tK8Nk13Q8Gc>" %}
Manually Launch an Analysis in ICA
{% endembed %}
{% endtab %}

{% tab title="Requeue Analysis in BaseSpace" %}

#### Requeue Analysis in BaseSpace

The secondary analysis for a sequencing run may need to be re-executed for a variety of reasons including:

* Fix a Sample Sheet error
* Intermittent analysis failure

The operation to manually relaunch the secondary analysis is referred to as "requeue". Requeues are performed from BaseSpace Sequence Hub.

{% hint style="info" %}

* This instruction below can be used to requeue auto-launch analysis for NovaSeq X/X Plus, NovaSeq 6000, NextSeq 500/550, NextSeq 1000/2000.
* **Requeue > Sample Sheet** will load the original sample sheet and can be modified directly in the browser. This is for advanced users and less validation of the sample sheet will be performed than if using **Requeue > Planned Run.**
  {% endhint %}

To requeue autolaunch analysis in the BaseSpace UI, follow the steps:

1. Click on your run in the "Runs" tab. From the Run Summary Page, select **Status > Requeue > Planned Run.**\\

   <figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-1fe9016dd8a8968c9d443ab5aa823e0b49b06311%2Fimage%20(10)%20(1)%20(1).png?alt=media" alt=""><figcaption></figcaption></figure>

Upload your new, corrected Sample Sheet. This should be the v2 sample sheet that is used for autolaunch.

<figure><img src="https://3223063594-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YNCzdEu5yaFoMebBG1p%2Fuploads%2Fgit-blob-8cbafac6e2f6cf916c9072a893467df822fe7349%2Fimage%20(6)%20(1)%20(1)%20(1).png?alt=media" alt=""><figcaption></figcaption></figure>
{% endtab %}
{% endtabs %}

## Using API to Launch Analysis

There may be situations when an analysis needs to be launched manually with the ICA API if the auto-launched analysis failed. Use the API to create an analysis and **set the input type to basespace** (for example `/api/projects/{projectId}/analysis:nextflow` with `analysisInput > inputs > externalData > type:basespace`) This works for both files and folders.

```
{
  "externalData": [
    {
      "url": "https://api.servername.illumina.com/v2/files/your_files",
      "type": "basespace",
      "mountPath": "/location_where_input_file_is_saved_on_machine_running_pipeline",
      "basespaceDetails": {
        "workgroupId": "wid:123" (Optional workgroup id to filter on)
        "extensions": "vcf" (Optional BSSH API query param to filter based on file extensions)
        "pathPrefix": "/path/to/files" (Optional BSSH query string param to filter files/folder)
      }
    }
  ]
}
```

## Secondary Analysis Troubleshooting

Secondary analysis failures may happen for a variety of reasons. When an error occurs, the Sequencing Run in BaseSpace will indicate a failure with the analysis. Depending on the specific failure, the workflow session and/or the orchestrated analysis in ICA will indicate a "Failed" status with an Error output in the details view.

Staring with BaseSpace 7.29, there is now improved error reporting through BaseSpace. A link in the logs section of the Analysis Summary tab opens a modal displaying a list of failed steps and relevant log file contents.

### Analyses Not Showing Up in ICA

* If no BaseSpace-managed runs appear in ICA, verify in [IAM](https://help.ica.illumina.com/account-management/am-iam) that the workgroup used in BaseSpace **has access** to ICA.
* Ensure that the option **Has Access** is selected in [IAM](https://help.ica.illumina.com/account-management/am-iam) as the option **Has Access+Archive** has been decommissioned