The Illumina MiSeqDx Integration Package v1.10.0 supports the integration between Clarity LIMS and MiSeqDx instruments.
This documentation describes the integration between Clarity LIMS and the MiSeqDx system. It includes information about protocols and automations, configuration options, installed components, and rules and constraints.
The following protocols are included in MiSeqDx Integration Package v1.10.0:
CF 139-Variant Assay Library Prep 1.2
CF Clinical Sequencing Assay Library Prep 1.2
Illumina SBS MiSeqDx (CF 139-Variant Assay) 1.2
Illumina SBS MiSeqDx (CF Clinical Sequencing Assay) 1.2
Illumina SBS MiSeqDx (Universal Kit) 1.2
Universal Kit Library Prep 1.2
There are three validation protocols. Each protocol is included in a workflow with the same name. The protocols are as follows.
MiSeqDx Validation (CF 139-Variant Assay) 1.2
MiSeqDx Validation (CF Clinical Sequencing Assay) 1.2
MiSeqDx Validation (Universal Kit) 1.2
In Assay workflows, include a QC validation protocol between the Library Prep and Illumina SBS protocols. The default workflow does not include a QC protocol.
Each validation protocol includes the following steps from the Library Prep and Illumina SBS MiSeqDx protocols:
Extension-Ligation of Bound Oligos (Library Prep step)
PCR Amplification (Library Prep step)
Library Pooling (MiSeqDx) (Illumina SBS MiSeqDx step)
Denature, Dilute and Load Sample (Illumina SBS MiSeqDx step)
MiSeqDx Run (MiSeqDx) (Illumina SBS MiSeqDx step)
Variant Calling (MiSeqDx) (Illumina SBS MiSeqDx step)
For instructions on user interaction for each step and using the MiSeqDx Validation (CF 139-Variant Assay) 1.2 protocol to validate automated sample sheet generation, refer to MiSeqDx Integration v1.10.0 User Interaction, Validation and Troubleshooting.
The following table lists the automations included in this integration, and the steps on which they are configured.
MiSeqDx Integration Default Automations
This section discusses the index placement and validation automations configured on the PCR Amplification 1.2 Library Prep steps.
The example workflow uses the CF 139-Variant Assay Library Prep 1.2 protocol.
By default, in the CF 139-Variant Assay Library Prep 1.2 and the Universal Kit Library Prep 1.2 protocols, the PCR Amplification 1.2 step includes two automations. Both automations invoke the place_indexes script with different options.
Auto Place Indexes — Automatically invoked on entry to the Add Reagents screen.
Validate Index Placement — Automatically invoked on exit from the Add Reagents screen.
A list of reagents installed with this configuration is provided in Installed Components.
This section describes the features of the key steps in the Illumina SBS MiSeqDx v1.2 protocols.
Library normalization CSV file generation in Library Normalization (MiSeqDx) step.
Reagent cartridge name validation and sample sheet generation in Denature, Dilute and Load Sample 1.2 step.
Primary analysis (sequencing) results parsing, which includes generation of the run report, in MiSeqDx Run (MiSeqDx) step.
Secondary analysis results parsing in Variant Calling (MiSeqDx) 1.2 step.
In each of the Illumina SBS MiSeqDx protocols, the Library Normalization (MiSeqDx) v1.2 step includes automated calculation of normalization buffer volumes. The results are provided in an autogenerated comma-separated file that is attached to the step.
In each of the Illumina SBS MiSeqDx and Validation protocols, the Denature, Dilute and Load Sample 1.2 step includes automations to:
Validate single input
Validate the MiSeqDx reagent cartridge name
Generate sample sheet
The integration allows for generation of a sample sheet designed to be used with the MOS (Diagnostic mode) instrument software.
By default, one MiSeqDx instrument sample sheet is created for the reagent cartridge loaded during the step. The placeholders and sample sheet files in the following table are generated by the Denature, Dilute and Load Sample 1.2 step.
Generated Placeholders and Sample Sheet Files
The MiSeqDx Run (MiSeqDx) step records information for each lane of the flow cell and generates a report summarizing the results. In addition, run parameters, run info, and a link to the run folder are automatically captured.
The following table describes the run information files, reports, placeholders, and links that Clarity LIMS automatically generates or captures during a sequencing run.
Run Information Generated or Captured by MiSeqDx Run (MiSeqDx) v1.2 Step
The following metadata are stored as custom fields tagged to the MiSeqDx Run (MiSeqDx step). These fields are all read-only.
Finish Date* – run completion date
Run Type – Diagnostic or Research mode
Status – current status of the sequencing run on the instrument
Flow Cell ID
Flow Cell Version
Experiment Name – entered in software
Read 1 Cycles
Index 1 Read Cycles – intended Index cycles
Index 2 Read Cycles – intended Index cycles
Read 2 Cycles
Output Folder – run folder root
Run ID – the unique run ID
Reagent Cartridge ID
Reagent Cartridge Part #
PR2 Bottle ID
Chemistry
Workflow
There are two additional master step fields in this step:
Comments — multiline text field used for any comments that are attached to the run
Report Status — hidden read-only field that tracks if the run report has been successfully uploaded
If the End Run event contains a date in the format YYYY-MM-DD, Finish Date will be set to the date in the event file.
If the End Run event does not contain a date or the date is in the wrong format, Finish Date will be set to the date when the event file is processed.
The following table lists the Real-Time Analysis (RTA) primary analysis metrics Clarity LIMS automatically captures and records, per read, for samples in each flow cell lane. These metrics are captured upon run completion.
RTA Primary Analysis Metrics Captured by MiSeqDx Run (MiSeqDx) Step
The sequencing service may run on Clarity LIMS server in an on-premise environment or on a remote server in a SaaS environment. The service detects event files that the instrument software (RTA) is producing as the run progresses. These event files let the service know where to find the run data.
As the run data are written out and the events come in (Begin Run, Cycle Complete, End Run), the data are matched to the step based on the reagent cartridge ID. This value was entered as the Container Name on the Denature, Dilute and Load Sample 1.2 step. The read-only field values on the Record Details screen are populated accordingly.
The Variant Calling (MiSeqDx) 1.2 step attaches secondary analysis output files for each of the samples.
In addition, a Combined Variant Call File, a text file, and a variant call file for each individual sample are automatically captured following the completion of secondary analysis.
Following primary analysis, secondary analysis runs on the MiSeqDx instrument. Clarity LIMS detects the completion of secondary analysis and lets the service know where to find the result files. While the EndRun event is processed successfully for primary analysis, the sequence service generates a BeginSecondaryAnalysis event file in the same directory with the miseqdx.seqservice.eventFileDirectory property.
An example of how Clarity LIMS shows the location of the secondary analysis event file is:
This event file signals that secondary analysis has started. The sequence service periodically checks if the analysis is complete by looking for the CompletedJobInfo.xml file. This file is in the folder specified by the runNetworkLocation parameter of the event file.
The frequency of checking is defined by the miseqdx.seqservice.synchronizationPeriod property.
If the completion file appears, the sequence service processes the event. The event is archived in a similar way to how the EndRun event file is handled. If the event completion does not appear within the number days specified by the miseqdx.seqservice.ignoreUnmatchedContainerIdsWaitDays property, the service stops monitoring the event file and the file is archived.
The following table describes the Variant Call Format (VCF) files that Clarity LIMS captures and attaches to the step after the secondary analysis.
Variant Call Format Files Captured by the LIMS After Secondary Analysis
The Illumina MiSeqDx Integration RPM package installs the following components.
This integration requires installation of the associated NGS Extensions Package (refer to Release Notes).
The following table lists the properties installed with the Illumina MiSeqDx Integration Package. The following constraints are present when using the properties:
Sequencing runs are matched using the flow cell ID and the base name of the sequencing step – MiSeqDx Run (MiSeqDx).
Do not change this name – it is expected by the sequencing service that captures instrument run results. The base name is stored in the sequenceProcessBaseName property shown in Table 2. If this name is changed without the property being updated, the 'flow cell ID <-> sequencing step base name' matching system will fail.
If necessary, you may modify the step name by editing or adding to the text after the base name portion. This part of the text is not used in the matching system. For example, you could change MiSeqDx Run (MiSeqDx) 1.2 to MiSeqDx Run (MiSeqDx) v1.2.
Several additional properties, each with the ‘99’ suffix appended to their name, are also installed. These properties are intended for use by the Clarity LIMS support team in automated validation tests and are not listed in the table.
Properties Installed with the Illumina MiSeqDx Integration Package
It is possible to configure support for multiple, identical seqservice.netPathPrefixSearch property values. For details, refer to Configure Multiple Identical netPathPrefixSearch Values.
The following reagent categories/label groups are included in the default configuration for the MiSeqDx Integration Package:
CF 139-Variant Assay Indexes
CF 139-Variant Assay 8-Sample Indexes
CF Clinical Sequencing Assay 8-Sample Indexes
Universal Kit Indexes
Universal Kit 8-Sample Indexes
The following reagent kits are included in the default configuration for the Illumina MiSeqDx Integration Package:
CF 139-Variant Assay-Oligo Pool
CF Clinical Sequencing Assay-Oligo Pool
Custom Oligo Pool
Elution Buffer
EtOH
Extension-Ligation Mix
Hybridization Buffer
Index Primers
Library Beads
Library Dilution Buffer
Library Normalization Diluent
Library Normalization Wash
Library Storage Buffer
MiSeqDx Flow Cell - CF 139-Variant Assay
MiSeqDx Flow Cell - CF Clinical Sequencing Assay
MiSeqDx Flow Cell - Universal kit
MiSeqDx SBS Solution (PR2) - CF 139-Variant Assay
MiSeqDx SBS Solution (PR2) - CF Clinical Sequencing Assay
MiSeqDx SBS Solution (PR2) - Universal Kit
NaOH
PCR Clean-Up Beads
PCR Master Mix
PCR Polymerase
Stringent Wash Buffer
Universal Wash Buffer
The following controls are included in the default configuration for the MiSeqDx Integration Package:
Negative Control for MiSeqDx
PhiX Internal Control
Positive Control for MiSeqDx
All one-dimensional container types with both numeric rows and numeric columns are supported.
The following container types are included in the default configuration for the MiSeqDx Integration Package:
MiSeqDx Reagent Cartridge - CF 139-Variant Assay
MiSeqDx Reagent Cartridge - CF Clinical Sequencing Assay
MiSeqDx Reagent Cartridge - Universal Kit
The following information provides an overview of the steps performed by the Clarity LIMS support team when configuring the instrument for use with the MiSeqDx Integration to Clarity LIMS.
Configure the MiSeqDx as follows.
Create a directory on the local computer to hold the batch files. These batch files write event files to the network-attached storage (NAS) shares.
Create a directory on the NAS to hold the event files.
Modify the software configuration files to call the batch files that create the event files.
Update sequencing service default properties to match the specifics of the installation.
This integration operates with the following constraints:
There must only be a single input on the Denature, Dilute and Load Sample 1.2 step.
The Reagent Cartridge ID must be unique. There should not be multiple containers in the system with the same name.
The reagent cartridge ID must be scanned as the Container Name on the Denature, Dilute and Load Sample 1.2 step.
For MOS sample sheet generation constraints, refer to Illumina MiSeqDx 1.10.0 Sample Sheet Generation.
For the Auto Place Indexes automation, if an 8-sample reagent category/label group is selected, index placement must be performed manually.
The CF Clinical Sequencing Assay Library Prep 1.2 protocol only uses 8-sample reagent categories/label groups. Hence, index placement must be performed manually.
Good laboratory practices mandate that a positive control DNA sample and a negative (no-template) control sample are included in every run. The positive control DNA sample should be a well-characterized sample with a known CFTR mutation.
To maintain unique values in the Sample_Name column of the sample sheet, the sample sheet generation automation script has the --useSampleLimsID parameter set to true. The --appendLimsID parameter must not be set to true. For the automation to display the correct instrument name when it is completed, the --isMiSeqDx parameter must be set to true.
Date Run is populated with the date that the Begin Run event file is first picked up and associated with a step in Clarity LIMS, not the date of the Begin Run event itself (the date the run was performed on the instrument). This property is provided by default in Clarity LIMS and is hidden. It is used in the AUTOMATED - Run Report Generation automation. Finish Date is populated as follows.
It is not intended for the user to select the AUTOMATED - Run Report Generation button. The run report is generated and attached to the step automatically.
Following the sequencing run, the run report is automatically generated by the AUTOMATED - Run Report Generation automation. Do not change this name. This name is expected by the sequencing service that captures instrument run results. The base name is stored in the sequenceProcessBaseName property. If this name is changed without the property being updated, the 'flow cell ID <-> sequencing step base name' matching will fail.
Changes on miseqdx.seqservice.sequenceProcessBaseName and miseqdx.seqservice.variantCallingProcessBaseName properties take effect upon updates and do not require restart of the integration service. For all remaining properties, integration service has to be restarted for property changes to take effect.
To make sure that your Illumina instrument warranty remains valid, the instrument integration must be performed and maintained by the Clarity LIMS support team. To perform this integration, the Clarity LIMS support team will require direct access to the instrument via WebEx or Remote Desktop while the instrument is idle.
Protocol
Step
Automations on Step
CF 139-Variant Assay Library Prep 1.2
Universal Kit Library Prep 1.2
PCR Amplification (CF 139-Variant Assay/Universal Kit) 1.2
Auto Place Indexes
Validate Index Placement
Illumina SBS MiSeqDx (CF 139-Variant Assay) 1.2
Illumina SBS MiSeqDx (CF Clinical Sequencing Assay) 1.2
Illumina SBS MiSeqDx (Universal Kit) 1.2
Library Normalization (MiSeqDx) 1.2
Create Normalization CSV
Illumina SBS MiSeqDx (CF 139-Variant Assay) 1.2
Illumina SBS MiSeqDx (CF Clinical Sequencing Assay) 1.2
Illumina SBS MiSeqDx (Universal Kit) 1.2
MiSeqDx Validation (CF 139-Variant Assay) 1.2
MiSeqDx Validation (CF Clinical Sequencing Assay) 1.2
MiSeqDx Validation (Universal Kit) 1.2
Denature, Dilute and Load Sample (CF 139-Variant Assay / CF Clinical Sequencing Assay / Universal Kit) 1.2
Generate MiSeqDx Sample Sheet
Validate Container Name
Validate Single Input
Illumina SBS MiSeqDx (CF 139-Variant Assay) 1.2
Illumina SBS MiSeqDx (CF Clinical Sequencing Assay) 1.2
Illumina SBS MiSeqDx (Universal Kit) 1.2
MiSeqDx Validation (CF 139-Variant Assay) 1.2
MiSeqDx Validation (CF Clinical Sequencing Assay) 1.2
MiSeqDx Validation (Universal Kit) 1.2
MiSeqDx Run (MiSeqDx) 1.2
AUTOMATED - Run Report Generation
Verify Report Status
Item
Description
Lab Tracking Form
This item allows you to manually attach a lab-specific tracking form to the step.
MiSeqDx Sample Sheet file
This CSV file is automatically generated by the LIMS for use with the MOS or MCS software.
May be opened as a text file, or as an MS Excel spreadsheet.
MiSeqDx Sample Sheet Generation Log
Automatically generated by Clarity LIMS, this log file captures any errors encountered when generating the MiSeqDx sample sheet.
Item
Description
Run Info Run Parameters
These XML files are automatically captured by Clarity LIMS from the instrument run folder. They include the key run parameters, many of which are parsed out into key master step and global fields.
Link to Run Folder
Automatically generated by Clarity LIMS, this is a link to the network run folder where the data that was captured from the instrument during the run is stored.
Illumina Run Report
Automatically generated by Clarity LIMS, this report provides key information about the run and the samples on the flow cell.
Information includes the flow cell ID, run directory location, instrument name, as well as primary analysis metrics for the instrument – summarized per flow cell lane for the entire run, and individual reads in the case of paired-end runs.
These metrics are compared against the instrument's per lane averages, calculated using metrics from the last five sequencing runs. Any values outside of one standard deviation are highlighted.
The run report calculations assume that the flow cell has 8 lanes. If you are using a two-lane rapid-run flow cell, the calculated values will be incorrect.
Lab Tracking Form
This item allows you to manually attach a lab-specific tracking form to the step.
RTA Primary Analysis Metrics
Corresponding Fields
Raw Yield (Gb)
Yield PF (Gb) R1 Yield PF (Gb) R2
% Bases >Q30
% Bases >=Q30 R1 % Bases >=Q30 R2
Cluster Density (K/mm^2)
Cluster Density (K/mm^2) R1 Cluster Density (K/mm^2) R2
Clusters Raw
Clusters Raw R1 Clusters Raw R2
Clusters PF
Clusters PF R1 Clusters PF R2
%PF
%PF R1 %PF R2
Intensity Cycle 1
Intensity Cycle 1 R1 Intensity Cycle 1 R2
% Intensity Cycle 20
% Intensity Cycle 20 R1 % Intensity Cycle 20 R2
% Phasing
% Phasing R1 % Phasing R2
% Prephasing
% Prephasing R1 % Prephasing R2
% Aligned
% Aligned R1 % Aligned R2
% Error Rate
% Error Rate R1 % Error Rate R2
File Type
Description
Combined Variant Call File
Variant call VCF files for all samples are compressed into a zip file and attached to the step for every run. Available for all 3 assays.
Combined Output Text File
For CF Clinical Sequencing Assay and CF 139-Variant Assay only, a text file containing the summary of variant call information for all samples is attached for every run.
Variant Call File per sample
The general variant call VCF file (not CFTR VCF file) for each sample is attached. Available for all 3 assays.
Files Installed
Location
Description
configure_extensions_miseqdx_workflow.sh
/opt/gls/clarity/config/
Script that imports the integration configuration.
miseqdx-extensions.jar
/opt/gls/clarity/extensions/miseqdx
Jar file containing API-based LIMS extensions used throughout the protocols.
configure_extensions_miseqdx_sequencingservice.sh
/opt/gls/clarity/config/extensions/miseqdx
Script that imports the properties for the integration.
log4j2.xml
/opt/gls/clarity/extensions/miseqdx/SequencingService/conf
File containing the settings for the sequencing jar logging.
miseqdx-sequencing-report.jar
/opt/gls/clarity/extensions/miseqdx/
Jar file containing API-based LIMS extensions used for report generation.
miseqdx-sequencing.jar
/opt/gls/clarity/extensions/miseqdx/SequencingService
Sequencing service jar file that captures the run results.
miseqdx_seqservice
/opt/gls/clarity/extensions/miseqdx/SequencingService/bin
Provides commands for interacting with the sequencing service. Accessible using systemctl. For example, use the following command to start the sequencing service:
The start keyword can be replaced with stop or status to operate on the sequencing service
run_miseqdx_seqservice
/opt/gls/clarity/extensions/miseqdx/SequencingService/bin
Called by miseqdx_seqservice-v1, this script performs some basic system validation, then starts the sequencing service.
MiSeqDxIntegrator.log
/opt/gls/clarity/extensions/miseqdx/SequencingService
Log file for Sequencing service.
Property
Description
Default Value
miseqdx.seqservice.ignoreUnmatchedContainerIds
A flag indicating if event files that cannot be matched to flow cells in Clarity LIMS should be archived after a period of time (true), or continually reprocessed (false).
false
miseqdx.seqservice.ignoreUnmatchedContainerIdsWaitDays
The number of days between when the event is created and the event file is archived.
14
miseqdx.seqservice.synchronizationPeriod
Invocation period in seconds.
60
miseqdx.seqservice.sequenceProcessBaseName
Sequencing process type base display name. Partial matching is used to look up the process type.
MiSeqDx Run (MiSeqDx)
miseqdx.seqservice.eventFileDirectory.1
A network location monitored for event files.
/mnt/gls_events
miseqdx.seqservice.netPathPrefixSearch.1
The network directory prefix contained in the event file.
\\nas\network\run_data
miseqdx.seqservice.netPathPrefixReplace.1
The mapped network directory mount name on the server used to access the run data directory.
/mnt/run_data
miseqdx.seqservice.eventFileDirectorySuffixes
A list of eventFileDirectory path entries to monitor for event files. The value is one or more comma-separated integers.
99
miseqdx.seqservice.runReportViewsVersion
The current version of the Run Report views in the database. The value 0 represents the state before the views are created. This property is automatically updated by the run report.
0
miseqdx.seqservice.netPathPrefixSearchReplaceSuffixes
A list of netPathPrefix search and replace entries for transforming Windows to Linux network paths. The value is one or more comma-separated integers.
99
miseqdx.seqservice.variantCallingProcessBaseName
Sequencing process type base display name. Partial matching is used to look up the process type.
Variant Calling (MiSeqDx)
To prevent the gls_events file directory from becoming cluttered, it is recommended that the value of this property is set to ‘true’.
