Illumina Support Center

Cloud Analysis Auto-launch

Cloud analysis auto-launch refers to the automated launch of secondary analysis by Illumina's cloud platforms after a sequencing run is completed. Coupled with the ability to automatically output sequencing data to a cloud location, the auto-launch functionality minimizes user touchpoints and decreases turnaround time.

Sequencing run management, including run planning, run monitoring, and viewing results, is performed via BaseSpace Sequence Hub (BaseSpace). The post-sequencing secondary analysis (ie, demultiplexing, DRAGEN) is powered by ICA pipelines. The analysis results are stored in an /ilmn-analyses/<sequencer_run_output> subfolder in ICA. See the Sequencer Auto-launch Analyses Compatibility table for available instruments and pipelines.

  • BaseSpace is used for sequencing run planning and monitoring. See BaseSpace Setup for more information.

  • ICA is used to store sequencing run data and perform secondary analysis. See Sequencer Run Data in ICA for more information.

  1. Run Planning - performed in BaseSpace to configure a sequencing run including choosing instrument type, analysis, and sample settings

  2. Start Sequencing Run - performed on the sequencing instrument to select the planned run and launch the run

  3. Data Upload and Run Monitoring

    • Data Upload - the sequencing instrument uploads the sequencing run output data to ICA

    • Run Monitoring - sequencing run monitoring is available in BaseSpace

  4. Auto-launch Analysis - the secondary analysis is automatically launched in ICA upon sequencing run completion

  5. Auto-launch Analysis Monitoring - the secondary analysis status is monitored in BaseSpace

  6. Edit & Requeue SampleSheet to trigger #4 (optional) - if an error occurs during secondary analysis, the analysis may be requeued (re-launched) from BaseSpace

Further details about each step above is described in the sections below.

BaseSpace Setup

BaseSpace settings must be configured to send sequencing run data to ICA prior to sequencing. For workgroups, this must be configured by the workgroup administrator. Workgroup users have read-access to the toggle switch setting, but cannot modify it.

Navigate to the BaseSpace Settings view. In the "Settings" section, set the "ICA Run Storage" setting to ON.

This setting is not necessary for NovaSeq X Series instrument type

The visibility of the "ICA Run Storage" configuration setting is controlled by Illumina support. Contact Illumina Support to request the setting be made available for your domain.

Sequencing Run Planning

Sequencing run planning in BaseSpace Sequence Hub is the first step in preparing a sequencing run with automated post-sequencing analysis.

A sequencing run can be started from the instrument using either:

  1. Planned sequencing run setup in Basespace Run Planning (recommended). See Plan the Sequencing Run for a guided example.

  2. Sample Sheet imported during the on-instrument Run Setup. See Cloud Analysis Auto-launch Settings for more information.

Refer to Sequencer Autolaunch Analyses Compatibility for guidance on sequencer compatibility with different versions of analyses applications.

Sequencing Run Monitoring in BaseSpace

Monitoring the status of an ongoing sequencing run is done through BaseSpace. See the View Runs BaseSpace help documentation for more information.

  1. Find the sequencing run in BaseSpace Sequence Hub by logging in, selecting the Runs tab, and selecting the appropriate sequencing run from the list.

  2. When the sequencing run completes uploading, the run status in BaseSpace Sequence Hub will move to Pending Analysis. Analysis will automatically launch based on the information in the Sample Sheet.

    1. If there is an issue with the Sample Sheet, the status will move to Needs Attention. Use the Requeue feature to correct any issues and re-launch analysis_._ See Analysis Requeue in BaseSpace.

  3. When the analysis launches, the run status will move to Analyzing. The associated analysis can be found on the Run Details tab under Latest Analysis, or in the Analysis list.

  4. Click the analysis link to go to the Analysis Details page. Information about the analysis progress will be displayed in the Logs section.

    1. If the analysis aborts with an error, logs and any files that had been produced prior to the error will available in the Files tab.

  5. When the analysis completes, any aggregate and/or sample reports can be viewed in the Reports tab. Logs and other output files can be viewed in the Files tab.

Sequencer Run Data in ICA

Sequencer run data is stored in ICA in an externally-managed project created by BaseSpace. Operations in BaseSpace, including planning and monitoring instrument runs, are performed in a given user or workgroup context. To preserve permission boundaries between applications, an externally-managed project is created for each user/workgroup context per region. The ICA project will be named using the convention BSSH {context name}, where {context-name} is the user or workgroup name operating in BaseSpace. For workgroups, the project will be created on behalf of the workgroup owner. The project Team includes an entry for the workgroup to grant all workgroup users read-only access to the project. While Team members may be assigned the Contributor role in the externally-managed project, they will not have write access (ie, create/delete data, run pipelines) in the project.

In the example below, the project is created for a workgroup with name "Research" where the workgroup owner is "bsadmin@illumina.com".

The project contains the following tags:

  • Technical Tags: bssh.project.user:{user/workgroup ID}

For workgroups, users who are members of the workgroup will have access to the data in the externally-managed project.

All sequencing run data is stored in the ilmn-runs folder. Within that folder, the data for each sequencing run is stored in a sub-folder named based on the run name used to create the run during run planning (or as specified in the RunName row of the [Header] section in the sample sheet) appended with a unique identifier.

The screenshot below provides an example of a sequencer run output folder uploaded to ICA.

Sequencer run data uploaded directly to ICA via CLI or Service Connector to a user-managed project will not be available in BaseSpace.

Sequencer run data stored outside of ICA or BaseSpace may be uploaded using the BaseSpace CLI (command-line interface) to make it available in both BaseSpace and ICA.

Use the BaseSpace Uploading a run instructions to upload a sequencer run output folder.

Automated Secondary Analysis in ICA

When the sequencing run data upload has completed, the post-sequencing secondary analysis described in the sample sheet will be automatically started.

In ICA, secondary analyses launched after a sequencing run completes is orchestrated by a parent process called a Workflow Session. The workflow session serves as a parent process orchestrating the secondary analysis, including launching secondary analysis pipelines and creating sample entities in ICA.

Upon completion of the sequencing run data upload, a workflow session record is created and visible in the ICA "Analyses" view. The workflow session record can be identified by the User Reference, which will have "ws" preceding the run name.

The workflow session details view includes a section Orchestrated Analyses to indicate analyses launched by the workflow session as part of the automated secondary analysis. These orchestrated analyses are driven by the information in the sample sheet, including the pipeline to launch, reference data, and sample-specific settings.

Currently, only a limited set of Illumina-provided pipelines are compatible with workflow session automation.

Analysis Requeue in BaseSpace

The secondary analysis for a sequencing run may need to be re-executed for a variety of reasons including:

  • Fix a Sample Sheet error

  • Intermittent analysis failure

The operation to manually relaunch the secondary analysis is referred to as "requeue". Requeues are performed from BaseSpace Sequence Hub.

  • This instruction below can be used to requeue autolaunch analysis for NovaSeq X/X Plus, NovaSeq 6000, NextSeq 500/550, NextSeq 1000/2000.

  • Do not use the Requeue > Sample Sheet option to requeue an auto-launched analysis since this step is ONLY used for v1 sample sheet requeue. It is not intended to use to requeue an auto-launch analysis on ICA.

BaseSpace must be configured to leverage ICA for run storage. Navigate to the BaseSpace Settings view by clicking your name/workgroup at the top right and selecting "Settings". In the "Settings" section, set the "ICA Run Storage" setting to ON. For workgroups, this must be configured by the workgroup administrator and configured in the correct workgroup.

To requeue autolaunch analysis in the BaseSpace UI, follow the steps:

  1. Click on your run in the "Runs" tab. From the Run Summary Page, select Status > Requeue > Planned Run.

  2. Upload your new, corrected Sample Sheet. This should be the v2 sample sheet that is used for autolaunch.

Manual Launch Analysis in ICA

There may be situations when an analysis needs to be launched manually in ICA, such as if the auto-launched analysis failed. Analyses can be launched manually in ICA with a few extra steps to prepare the input data.

For example, if the demultiplexing analysis fails due to a sample sheet mistake or a system error, the analysis can be launched manually in ICA by following these instructions:

  1. Create a new or choose an existing user-managed project to link the data to and run the analysis in

  2. Follow instructions to Link Project Data to link the sequencing run output data to the user-managed project data

The "ilmn-runs" folder contains all sequencer run data managed by BaseSpace and should not be linked to a user-managed project. Only link individual sequencer run output folders within the ilmn-runs folder. See Sequencer Run Data for details on how to find the correct folder to link.

  1. Link the Bundle containing the ICA pipeline to the user-managed project (ie, "DRAGEN Analysis 4.1.7 - Sequencer Integration Only") (See Link Bundles)

  2. Fix the sample sheet by downloading the original sample sheet from the sequencing run output folder, modify, and re-upload the file to the user-managed project

  3. Launch the pipeline (ie, BclConvert_v4_1_7) with the corrected sample sheet as input

Using API to Launch Analysis

There may be situations when an analysis needs to be launched manually with the ICA API if the auto-launched analysis failed. Use the API to create an analysis and set the input type to basespace (for example /api/projects/{projectId}/analysis:nextflow with analysisInput > inputs > externalData > type:basespace) This works for both files and folders.

{
  "externalData": [
    {
      "url": "https://api.servername.illumina.com/v2/files/your_files",
      "type": "basespace",
      "mountPath": "/location_where_input_file_is_saved_on_machine_running_pipeline",
      "basespaceDetails": {
        "workgroupId": "wid:123" (Optional workgroup id to filter on)
        "extensions": "vcf" (Optional BSSH API query param to filter based on file extensions)
        "pathPrefix": "/path/to/files" (Optional BSSH query string param to filter files/folder)
      }
    }
  ]
}

Secondary Analysis Troubleshooting

Secondary analysis failures may happen for a variety of reasons. When an error occurs, the Sequencing Run in BaseSpace will indicate a failure with the analysis. Depending on the specific failure, the workflow session and/or the orchestrated analysis in ICA will indicate a "Failed" status with an Error output in the details view.

PreviousAnalysis ProductsNextSequencer Auto-launch Analyses Compatibility

Last updated