Cloud Analysis Auto-launch
Last updated
Last updated
Cloud analysis auto-launch refers to the automated launch of secondary analysis by Illumina's cloud platforms after a sequencing run is completed. Coupled with the ability to automatically output sequencing data to a cloud location, the auto-launch functionality minimizes user touchpoints and decreases turnaround time.
Sequencing run management, including run planning, run monitoring, and viewing results, is performed via BaseSpace Sequence Hub (BaseSpace). The post-sequencing secondary analysis (ie, demultiplexing, DRAGEN) is powered by ICA pipelines. The analysis results are stored in an /ilmn-analyses/<sequencer_run_output> subfolder in ICA. See the Sequencer Auto-launch Analyses Compatibility table for available instruments and pipelines.
BaseSpace is used for sequencing run planning and monitoring. See BaseSpace Setup for more information.
ICA is used to store sequencing run data and perform secondary analysis. See Sequencer Run Data in ICA for more information.
Run Planning - performed in BaseSpace to configure a sequencing run including choosing instrument type, analysis, and sample settings
Start Sequencing Run - performed on the sequencing instrument to select the planned run and launch the run
Data Upload and Run Monitoring
Data Upload - the sequencing instrument uploads the sequencing run output data to ICA
Run Monitoring - sequencing run monitoring is available in BaseSpace
Auto-launch Analysis - the secondary analysis is automatically launched in ICA upon sequencing run completion
Auto-launch Analysis Monitoring - the secondary analysis status is monitored in BaseSpace
Edit & Requeue SampleSheet to trigger #4 (optional) - if an error occurs during secondary analysis, the analysis may be requeued (re-launched) from BaseSpace
Further details about each step above is described in the sections below.
BaseSpace settings must be configured to send sequencing run data to ICA prior to sequencing. For workgroups, this must be configured by the workgroup administrator. Workgroup users have read-access to the toggle switch setting, but cannot modify it.
Navigate to the BaseSpace Settings view. In the "Settings" section, set the "ICA Run Storage" setting to ON.
This setting is not necessary for NovaSeq X Series instrument type
The visibility of the "ICA Run Storage" configuration setting is controlled by Illumina support. Contact Illumina Support to request the setting be made available for your domain.
Sequencing run planning in BaseSpace Sequence Hub is the first step in preparing a sequencing run with automated post-sequencing analysis.
A sequencing run can be started from the instrument using either:
Planned sequencing run setup in Basespace Run Planning (recommended). See Plan the Sequencing Run for a guided example.
Sample Sheet imported during the on-instrument Run Setup. See Cloud Analysis Auto-launch Settings for more information.
Refer to Sequencer Autolaunch Analyses Compatibility for guidance on sequencer compatibility with different versions of analyses applications.
Monitoring the status of an ongoing sequencing run is done through BaseSpace. See the View Runs BaseSpace help documentation for more information.
Find the sequencing run in BaseSpace Sequence Hub by logging in, selecting the Runs tab, and selecting the appropriate sequencing run from the list.
When the sequencing run completes uploading, the run status in BaseSpace Sequence Hub will move to Pending Analysis. Analysis will automatically launch based on the information in the Sample Sheet.
If there is an issue with the Sample Sheet, the status will move to Needs Attention. Use the Requeue feature to correct any issues and re-launch analysis. See Requeue Analysis.
When the analysis launches, the run status will move to Analyzing. The associated analysis can be found on the Run Details tab under Latest Analysis, or in the Analysis list.
Click the analysis link to go to the Analysis Details page. Information about the analysis progress will be displayed in the Logs section.
If the analysis aborts with an error, logs and any files that had been produced prior to the error will available in the Files tab.
When the analysis completes, any aggregate and/or sample reports can be viewed in the Reports tab. Logs and other output files can be viewed in the Files tab.
Sequencer run data is stored in ICA in an externally-managed project created by BaseSpace. Operations in BaseSpace, including planning and monitoring instrument runs, are performed in a given user or workgroup context. To preserve permission boundaries between applications, an externally-managed project is created for each user/workgroup context per region. The ICA project will be named using the convention BSSH {context name}, where {context-name} is the user or workgroup name operating in BaseSpace. For workgroups, the project will be created on behalf of the workgroup owner. The project Team includes an entry for the workgroup to grant all workgroup users read-only access to the project. While Team members may be assigned the Contributor role in the externally-managed project, they will not have write access (ie, create/delete data, run pipelines) in the project.
In the example below, the project is created for a workgroup with name "Research" where the workgroup owner is "bsadmin@illumina.com".
The project contains the following tags:
Technical Tags: bssh.project.user:{user/workgroup ID}
For workgroups, users who are members of the workgroup will have access to the data in the externally-managed project.
All sequencing run data is stored in the ilmn-runs folder. Within that folder, the data for each sequencing run is stored in a sub-folder named based on the run name used to create the run during run planning (or as specified in the RunName row of the [Header] section in the sample sheet) appended with a unique identifier.
The screenshot below provides an example of a sequencer run output folder uploaded to ICA.
Sequencer run data uploaded directly to ICA via CLI or Service Connector to a user-managed project will not be available in BaseSpace.
Sequencer run data stored outside of ICA or BaseSpace may be uploaded using the BaseSpace CLI (command-line interface) to make it available in both BaseSpace and ICA.
Use the BaseSpace Uploading a run instructions to upload a sequencer run output folder.
When the sequencing run data upload has completed, the post-sequencing secondary analysis described in the sample sheet will be automatically started.
In ICA, secondary analyses launched after a sequencing run completes is orchestrated by a parent process called a Workflow Session. The workflow session serves as a parent process orchestrating the secondary analysis, including launching secondary analysis pipelines and creating sample entities in ICA.
Upon completion of the sequencing run data upload, a workflow session record is created and visible in the ICA "Analyses" view. The workflow session record can be identified by the User Reference, which will have "ws" preceding the run name.
The workflow session details view includes a section Orchestrated Analyses to indicate analyses launched by the workflow session as part of the automated secondary analysis. These orchestrated analyses are driven by the information in the sample sheet, including the pipeline to launch, reference data, and sample-specific settings.
Currently, only a limited set of Illumina-provided pipelines are compatible with workflow session automation.
There may be situations when an analysis needs to be launched manually in ICA, such as if the auto-launched analysis failed. Analyses can be launched manually in ICA with a few extra steps to prepare the input data.
For example, if the demultiplexing analysis fails due to a sample sheet mistake or a system error, the analysis can be launched manually in ICA by following these instructions:
Create a new or choose an existing user-managed project to link the data to and run the analysis in
Follow instructions to Link Project Data to link the sequencing run output data to the user-managed project data
The "ilmn-runs" folder contains all sequencer run data managed by BaseSpace and should not be linked to a user-managed project. Only link individual sequencer run output folders within the ilmn-runs folder. See Sequencer Run Data for details on how to find the correct folder to link.
Link the Bundle containing the ICA pipeline to the user-managed project (ie, "DRAGEN Analysis 4.1.7 - Sequencer Integration Only") (See Link Bundles)
Fix the sample sheet by downloading the original sample sheet from the sequencing run output folder, modify, and re-upload the file to the user-managed project
Launch the pipeline (ie, BclConvert_v4_1_7) with the corrected sample sheet as input
There may be situations when an analysis needs to be launched manually with the ICA API if the auto-launched analysis failed. Use the API to create an analysis and set the input type to basespace (for example /api/projects/{projectId}/analysis:nextflow with analysisInput > inputs > externalData > type:basespace) This works for both files and folders.
Secondary analysis failures may happen for a variety of reasons. When an error occurs, the Sequencing Run in BaseSpace will indicate a failure with the analysis. Depending on the specific failure, the workflow session and/or the orchestrated analysis in ICA will indicate a "Failed" status with an Error output in the details view.
Staring with BaseSpace 7.29, there is now improved error reporting through BaseSpace. A link in the logs section of the Analysis Summary tab opens a modal displaying a list of failed steps and relevant log file contents.
