The documents in this section support the integration of the Clarity LIMS Product Analytics service. This integration supports on premise and Illumina cloud hosted deployments of Clarity LIMS v6.0 and later. Illumina Preset Protocols v2.3 or later is required for automation.

The Clarity LIMS Product Analytics Integration v1.2.0 allows you to aggregate and analyze lab data from the workflow in a single location. The integration also integrates Clarity LIMS data into the Clarity LIMS Product Analytics (CLPA) service. Data are uploaded to the Illumina Connected Analytics (ICA) service.

Refer to individual integration package requirement on enabling UPA. Note that ICA subscription is only required to view data on ICA UI.

CLPA integration supports the workflows: NovaSeq 6000 v3, NextSeq 1000/2000, NovaSeq X Series and Library Prep Validation.

Other workflows can be customized to add Clarity LIMS Product Analytics integration capabilities. For more information, refer to Automation and Workflow Customization.

Clarity LIMS Product Analytics Integration v1.2.0 is used with Clarity LIMS v6.2 and later. The integration supports on-site and hosted installations of Clarity LIMS. Illumina Preset Protocols (IPP) v2.6 or later is required for automation.

Known Issues

• UPA payload generation at pooling step and sequencing step will fail if any of the samples in the pools has been assigned QC flag in prior steps before entering these steps.

Supported Workflows

Clarity LIMS Product Analytics Integration v1.2.0 supports the applicable workflows identified in Supported Workflows.

Components

Clarity LIMS Product Analytics Integration v1.2.0 includes the following required components:

• BaseSpaceLIMS-unified-products-analytics — The core service of the Clarity LIMS Product Analytics Integration. This service installs the daemon service that publishes messages to the product analytics through Illumina Connected Analytics.

• BaseSpaceLIMS-unified-products-analytics-scripts — The automation scripts of the Clarity LIMS Product Analytics Integration. If new workflows or new product analytics scripts are introduced, this component can be upgraded.

Automation Scripts

The Clarity LIMS Product Analytics Integration v1.2.0 automation scripts are installed through the BaseSpaceLIMS-unified-products-analytics-scripts RPM. These scripts process Clarity LIMS data and prepare data for upload to Illumina Connected Analytics. There are two types of CLPA scripts: common and workflow-specific.

Common Clarity LIMS Product Analytics Scripts

Common CLPA scripts can be used with any compatible workflows. The following lists and describes the common CLPA scripts.

Workflow-Specific Clarity LIMS Product Analytics Scripts

Workflow-specific scripts work with Illumina workflows. These scripts require a specific set of custom fields and a specific protocol order in the workflow. The following shows the workflow-specific scripts.

As many of the scripts share trigger location and style, they are combined into a single automation. Unless specified otherwise, the automations are applicable to all CLPA-enabled versions of the relevant workflow steps.

The Clarity LIMS Product Analytics Integration v1.3.1 allows you to aggregate and analyze lab data from the workflow in a single location. Data is uploaded to the Illumina Connected Analytics (ICA) service.

Refer to individual integration package requirement on enabling UPA. Note that ICA subscription is only required to view data on ICA UI.

Clarity LIMS Product Analytics Integration v1.3.1 includes the following changes:

• The novaseqxseries_associate_seqrun.groovy and nextseq1k2k_associate_seqrun.groovy scripts can now associate libraries to sequencing run when any sample in the pool(s) has been assigned QC flags prior to entering the AUTOMATED - Sequencing Run step in the workflow.

• The pooling.groovy script no longer fails when any samples in the pool(s) has been assigned QC flags prior to entering pooling steps in the workflow.

This integration also supports the following workflows:

• NovaSeq 6000 v3

• NextSeq 1000/2000

• NovaSeq X Series

• Library Prep Validation

Other workflows can be customized to add Clarity LIMS Product Analytics integration capabilities. For more information, refer to Automation and Workflow Customization.

Supported Workflows

Clarity LIMS Product Analytics Integration v1.3.1 supports the applicable workflows identified in Supported Workflows. These workflows require this integration due to fixes that supports CLPA generation of payload (for pooling step) and association of library to sequencing run when samples go through QC step in workflow:

• NextSeq 1000/2000 Sequencing v2.4 workflow and later

• NovaSeq X Series Sequencing v1.1 workflow and later

Components

Clarity LIMS Product Analytics Integration v1.3.1 includes the following required components:

• BaseSpaceLIMS-unified-products-analytics — The core service of the Clarity LIMS Product Analytics Integration. This service installs the daemon service that publishes messages to the product analytics through the ICA service.

• BaseSpaceLIMS-unified-products-analytics-scripts — The automation scripts of the Clarity LIMS Product Analytics Integration. If new workflows or new product analytics scripts are introduced, this component can be upgraded.

Automation Scripts

The automation scripts are installed through the BaseSpaceLIMS-unified-products-analytics-scripts RPM. These scripts process Clarity LIMS data and prepare data for upload to the ICA service. There are two types of CLPA scripts: common and workflow-specific. The installed scripts are the same as that of Clarity LIMS Product Analytics Integration v1.3.0

Last Updated: July 2024

Clarity LIMS Product Analytics supports the workflows listed in the following table.

All workflows are provided as a configuration slice outside of the IPP installation.

Supported Workflows

¹ The integration supports on-premise or hosted installations. The on-premise support is only available to the NovaSeq 6000 v3 workflow. The NextSeq 1000/2000 and NovaSeq X Series workflows support only hosted installations. Clarity LIMS v6.2 discontinues support for CentOS and instead supports Oracle Linux as the superseding operating system.

² The NovaSeq X Series v1.0.0 and v1.1.0 integrations is only compatible with BaseSpaceLIMS-unified-product-analytics-scripts v1.2.0.

³ The NovaSeq X Series v1.2.0 integration requires BaseSpaceLIMS-unified-product-analytics-scripts v1.2.1 and later.

⁴ The CLPA enabled workflows are provided as separate configuration slices outside of the IPP installation.

⁵ Requires CLPA v1.2.0 or earlier.

⁶ Requires CLPA v1.3.0 or later due to updates to the nextseq1k2k_analysisrun_completed.groovy script.

Known Issues

• UPA payload generation at pooling step and sequencing step will fail if any of the samples in the pools has been assigned QC flag in prior steps before entering these steps for UPA Scripts v1.2.0, v1.2.1 and v1.3.0.

Clarity LIMS Product Analytics (CLPA) Integration is provided to integrate and make available Clarity data to Illumina Connected Analytics (ICA). CLPA allows user to aggregate and analyze lab data from end-to-end workflow in a single location.

These data are uploaded through ICA. Subscription to ICA is required in order to enable CLPA service.

CLPA integration supports the workflows:

• NovaSeq 6000 v3

• NextSeq 1000/2000

• NovaSeq X Series

• Library Prep Validation.

Other workflows (e.g. User's own custom workflow and other Illumina library preparation workflow) can be customized to add CLPA integration capabilities. This guide provides instructions to configure workflow with CLPA integration.

Contact Illumina Support for help to enable UPA on these older workflows:

• NovaSeq Workflow v3.3, v3.5 and v3.7

• NextSeq 1000/2000 Sequencing v2.1 Workflow

Prerequisites

1. BaseSpaceLIMS-unified-product-analytics and BaseSpaceLIMS-unified-product-analytics-scripts are installed and configured

2. IPP v2.3 and above is installed and at least 1 supported workflow installed.

3. Submitted Sample UDF "Illumina Universal Sample Identifier" is required by all UPA automation scripts.

Universal Sample Identifier

The Universal Sample Identifier is a Submitted Sample custom field used by the Register Sample automation. The field assigns a unique identifier to a sample across Illumina workflows. This identifier is used by CLPA to associate the sample with the workflows and analysis that result from a sequencing run and analysis (eg, sample lineage and traceability). Enter the Universal Sample Identifier value during sample accessioning. If the identifier is not provided during sample accessioning, the Register Sample automation does not have access to this value at run time. The CLPA association between the samples and other analytics data is also restricted.

When you install any of the CLPA enabled workflows, the Illumina Universal Sample Identifier custom field is automatically created. If the CLPA enabled workflows are not installed, the custom field must be created manually. When using the Illumina Universal Sample Identifier, the recommended data format is

usi.<clarity_hostname>.<sampleId>

For more information on installing CLPA enabled workflows, contact Illumina Support.

Add the Illumina Universal Sample Identifier Submitted Sample custom field:

1. On the main menu, select Configuration, and then select the Custom Fields tab.

2. On the Global Fields tab, in the Submitted Sample list, select the plus icon to create a custom field.

3. Enter Illumina Universal Sample Identifier as the field name.

4. Set the following properties:

   • For Field Type, select Text.

   • For Required Field, select No.

   • For Read Only, select No.

5. Select Save.

Use Common CLPA Automation

Common CLPA automation scripts are used for workflow customization. Each of these automation scripts logs a message into a log file that is defined with the parameter -l. Because steps can have different log placeholder positions, you must update the -l parameter to reference the correct log file placeholder (E.g., {compoundOutputFileLuid1}).

The following automations can be used on any step.

• Step Started:

  Copy

  bash -l -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/unified-product-analytics/automation/unified-product-analytics-automation.jar script:executeUPAAutomationScript -i {stepURI:v2} -u {username} -p {password} -l {compoundOutputFileLuid1} -s 'com/illumina/upa/scripts/common/step_started.groovy'"

• Step Completed:

  Copy

  bash -l -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/unified-product-analytics/automation/unified-product-analytics-automation.jar script:executeUPAAutomationScript -i {stepURI:v2} -u {username} -p {password} -l {compoundOutputFileLuid1} -s 'com/illumina/upa/scripts/common/step_completed.groovy'"

The Register Sample automation can be used on the Library Prep step. If there are multiple steps in the Library Prep workflow, only add the Register Sample automation to the first step of the protocol. The script is as follows.

bash -l -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/unified-product-analytics/automation/unified-product-analytics-automation.jar script:executeUPAAutomationScript -i {stepURI:v2} -u {username} -p {password} -l {compoundOutputFileLuid0} -s 'com/illumina/upa/scripts/common/register_sample.groovy'

The Register Pool automation is available for the Pooling workflow. The script is as follows.

bash -l -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/unified-product-analytics/automation/unified-product-analytics-automation.jar script:executeUPAAutomationScript -i {stepURI:v2} -u {username} -p {password} -l {compoundOutputFileLuid1} -s 'com/illumina/upa/scripts/common/pooling.groovy'

Workflow Customization

Customize the workflow with CLPA automation as follows.

1. Identify the type of CLPA automation to be added along with the trigger location and style. For more information on CLPA automation types and trigger location and styles, refer to Automation Scripts in Configuration for the UPA version of interest.

2. Identify the protocol step to be customized.

3. Check for any existing automations on the triggers (eg, Step/Automatic upon entry or Step/Automatic upon exit). If there is an existing automation, modify the script as follows.

   1. Append the automation after the existing script with &&. Refer to the following examples.

      • Before append:

        Copy

        bash -l -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/"

      • After append:

        Copy

        bash -l -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/ \
        && /opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/unified-product-analytics/automation/unified-product-analytics-automation.jar \
        script:executeUPAAutomationScript -i {stepURI:v2} -u {username} -p {password} \
        -l {compoundOutputFileLuid0} -s 'com/illumina/upa/scripts/common/step_completed.groovy'"

   2. Update the -l parameter to identify the correct Log File placeholder location.

      {compoundOutputFileLuid0} in the example refers to the first file placeholder.

   3. If necessary, update the automation name to describe the new behavior.

   4. In the Automation Use section, enable the step.
   5. Save the automation.

4. If there is not an existing automation on the protocol step, check for an existing automation that has been configured as follows.

   1. Identify the automation that is being used (e.g., Register Step Started).

   2. Make sure that the log file setting for the existing script has the -I parameter. This parameter identifies the same Log File placeholder that is used with the step.

      • If the Log File location is the same, enable the step in the Automation Use section.

      • If the Log File location is not the same, create a separate automation and enable the step in the Automation Use section of the new automation.

   3. Save the automation.

5. Go to Configuration and select Lab Work.

6. Select the step and configure the Trigger Location/Style. Refer to Automation Scripts in Configuration for the UPA version of interest.

   After the automation is enabled, the automation configuration option becomes available on the Step configuration page.

7. Save the step.

The Clarity LIMS Product Analytics Integration v1.3.0 allows you to aggregate and analyze lab data from the workflow in a single location. Data is uploaded to the Illumina Connected Analytics (ICA) service.

Refer to individual integration package requirement on enabling UPA. Note that ICA subscription is only required to view data on ICA UI.

Clarity LIMS Product Analytics Integration v1.3.0 includes the following changes:

• The step_completed.groovy script no longer fails when there are zero outputs for a step, such as no output step and file placeholder.

• The nextseq1k2k_analysisrun_completed.groovy script removes the # of > = Q30 Bases (PF) and Mean Quality Score (PF) fields. These fields are no longer supported in the NextSeq 1000/2000 Sequencing v2.4 workflows.

• The configure_upa.sh script provides the default value during setup.

• Configurable properties are now in the database and can be updated with the omxProps-ConfigTool utility.

This integration also supports the following workflows:

• NovaSeq 6000 v3

• NextSeq 1000/2000

• NovaSeq X Series

• Library Prep Validation

Other workflows can be customized to add Clarity LIMS Product Analytics integration capabilities. For more information, refer to Automation and Workflow Customization.

Known Issues

• UPA payload generation at pooling step and sequencing step will fail if any of the samples in the pools has been assigned QC flag in prior steps before entering these steps.

Supported Workflows

Clarity LIMS Product Analytics Integration v1.3.0 supports the applicable workflows identified in Supported Workflows. The NextSeq 1000/2000 Sequencing v2.4 workflow and later requires this integration due to updates to the nextseq1k2k_analysisrun_completed.groovy script.

Components

Clarity LIMS Product Analytics Integration v1.3.0 includes the following required components:

• BaseSpaceLIMS-unified-products-analytics — The core service of the Clarity LIMS Product Analytics Integration. This service installs the daemon service that publishes messages to the product analytics through the ICA service.

• BaseSpaceLIMS-unified-products-analytics-scripts — The automation scripts of the Clarity LIMS Product Analytics Integration. If new workflows or new product analytics scripts are introduced, this component can be upgraded.

Automation Scripts

The Clarity LIMS Product Analytics Integration v1.3.0 automation scripts are installed through the BaseSpaceLIMS-unified-products-analytics-scripts RPM. These scripts process Clarity LIMS data and prepare data for upload to the ICA service. There are two types of CLPA scripts: common and workflow-specific.

Common Clarity LIMS Product Analytics Scripts

Common CLPA scripts can be used with any compatible workflows. The following lists and describes the common CLPA scripts.

Workflow-Specific Clarity LIMS Product Analytics Scripts

Workflow-specific scripts work with Illumina workflows. These scripts require a specific set of custom fields and a specific protocol order in the workflow. The following table shows the workflow-specific scripts.

As many of the scripts in this table share trigger location and style, they are combined into a single automation.