LogoLogo
Illumina KnowledgeIllumina SupportSign In
Clarity LIMS Software
  • Home
Clarity LIMS Software
  • Announcements
  • Clarity LIMS
    • Clarity & LabLink
  • API and Database
    • API Portal
      • REST
        • REST General Concepts
        • REST Web Services
        • HTTP Response Codes and Errors
        • XML UTF-8 Character Encoding
        • Requesting API Version Information
        • Viewing Paginated List Resources
        • Filtering List Resources
        • Working with User-Defined Fields (UDF) and Types (UDT)
        • Traversing a Genealogy
        • Working with Batch Resources
      • Getting Started with API
        • Understanding API Terminology (LIMS v5 and later)
        • API-Based URIs (LIMS v4 and later)
        • Development Prerequisites
        • Structure of REST Resources
        • The Life Cycle of a Sample: Stages Versus Steps
        • Integrating Scripts
      • Automation
        • Automation Triggers and Command Line Calls
        • Automation Execution Environment
        • Supported Command Line Interpreters
        • Automation Channels
        • Error Handling
        • Automation Tokens
          • Derived Sample Automation Tokens
          • Step Automation Tokens
          • Project Automation Tokens
        • Automation Testing
        • Troubleshooting Automation
      • Tips and Tricks
        • Accessing Step UDFs from a different Step
        • Obfuscating Sensitive Data in Scripts
        • Integrating Clarity LIMS with Upstream Sample Accessioning Systems
        • Creating Samples and Projects via the API
        • Displaying Files From an Earlier Step
        • Transitioning Output Artifacts into the Next Step
        • Determining the Workflow(s) to Which a Sample is Assigned
        • Standardizing Sample Naming via the API
        • Copying UDF Values from Source to Destination
        • Updating Preset Value of a Step UDF through API
        • Automating BCL Conversion
        • Finding QC Flags in Aggregate QC (Library Validation) via REST API
        • Setting the Value of a QC Flag on an Artifact
        • Creating Notifications When Files are Added via LabLink
        • Remote HTTP Filestore Setup
      • Cookbook
        • Get Started with the Cookbook
          • Tips and Troubleshooting
          • Obtain and Use the REST API Utility Classes
        • Work with EPP/Automation and Files
          • Automation Trigger Configuration
          • Process Execution with EPP/Automation Support
        • Work with Submitted Samples
          • Adding Samples to the System
          • Renaming Samples
          • Assigning Samples to Workflows
          • Updating Sample Information
          • Show the Relationship Between Samples and Analyte Artifacts (Derived Samples)
        • Work with Containers
          • Add an Empty Container to the System
          • Find the Contents of a Well Location in a Container
          • Filter Containers by Name
        • Work with Derived Sample Automations
          • Remove Samples from Workflows
          • Requeue Samples
          • Rearray Samples
        • Work with Process/Step Outputs
          • Update UDF/Custom Field Values for a Derived Sample Output
          • Rename Derived Samples Using the API
          • Find the Container Location of a Derived Sample
          • Traverse a Pooled and Demultiplexed Sample History/Genealogy
          • View the Inputs and Outputs of a Process/Step
        • Work with Projects and Accounts
          • Remove Information from a Project
          • Add a New Project to the System with UDF/Custom Field Value
          • Get a Project Name
          • Find an Account Registered in the System
          • Update Contact (User and Client) Information
        • Work with Multiplexing
          • Find the Index Sequence for a Reagent Label
          • Demultiplexing
          • Pool Samples with Reagent Labels
          • Apply Reagent Labels with REST
          • Apply Reagent Labels When Samples are Imported
          • Apply Reagent Labels by Adding Reagents to Samples
        • Working with User Defined Fields/Custom Fields
          • About UDFs/Custom Fields and UDTs
          • Performing Post-Step Calculations with Custom Fields/UDFs
        • Work with Processes/Steps
          • Filter Processes by Date and Type
          • Find Terminal Processes/Steps
          • Run a Process/Step
          • Update UDF/Custom Field Information for a Process/Step
          • Work with the Steps Pooling Endpoint
        • Work with Batch Resources
          • Introduction to Batch Resources
          • Update UDF/Custom Field Information with Batch Operations
          • Retrieve Multiple Entities with a Single API Interaction
          • Select the Optimal Batch Size
        • Work with Files
          • Attach a File with REST and Python
          • Attach Files Located Outside the Default File Storage Repository
          • Attach a File to a File Placeholder with REST
        • Work with Controls
          • Automated Removal of Controls from a Workflow
      • Application Examples
        • Python API Library (glsapiutil.py) Location
        • Scripts That Help Automate Steps
          • Route Artifacts Based Off a Template File
          • Invoking bcl2fastq from BCL Conversion and Demultiplexing Step
          • Email Notifications
          • Finishing the Current Step and Starting the Next
          • Adding Downstream Samples to Additional Workflows
          • Advancing/Completing a Protocol Step via the API
          • Setting a Default Next Action
          • Automatic Placement of Samples Based on Input Plate Map (Multiple Plates)
          • Automatic Placement of Samples Based on Input Plate Map
          • Publishing Files to LabLink
          • Automatic Pooling Based on a Sample UDF/Custom Field
          • Completing a Step Programmatically
          • Automatic Sample Placement into Existing Containers
          • Routing Output Artifacts to Specific Workflows/Stages
          • Creating Multiple Containers / Types for Placement
          • Starting a Protocol Step via the API
          • Setting Quality Control Flags
          • Applying Indexing Patterns to Containers Automatically
          • Assignment of Sample Next Steps Based On a UDF
          • Parsing Metadata into UDFs (BCL Conversion and Demultiplexing)
        • Scripts That Validate Step Contents
          • Validating Process/Step Level UDFs
          • Checking That Containers Are Named Appropriately
          • Checking for Index Clashes Based on Index Sequence
          • Validating Illumina TruSeq Index Adapter Combinations
        • Scripts Triggered Outside of Workflows/Steps
          • Repurposing a Process to Upload Indexes
          • Adding Users in Bulk
          • Moving Reagent Kits & Lots to New Clarity LIMS Server
          • Programatically Importing the Sample Submission Excel File
          • Generating an MS Excel Sample Submission Spreadsheet
          • Assigning Samples to New Workflows
        • Miscellaneous Scripts
          • Illumina LIMS Integration
          • Generating a Hierarchical Sample History
          • Protocol-based Permissions
          • Self-Incremental Counters
          • Generic CSV Parser Template (Python)
          • Renaming Samples to Add an Internal ID
          • Creating Custom Sample Sheets
          • Copying Output UDFs to Submitted Samples
          • Parsing Sequencing Meta-Data into Clarity LIMS
          • Submit to a Compute Cluster via PBS
          • Downloading a File and PDF Image Extraction
        • Resources and References
          • Understanding LIMS ID Prefixes
          • Container States
          • Useful Tools
          • Unsupported Artifact Types
          • Unsupported Process Types
          • Suggested Reading
          • API Training Videos
  • Illumina Preset Protocols
    • IPP v2.10
      • Release Notes
      • Installation and User Configuration
      • Manual Upgrade
    • IPP v2.9
      • Release Notes
      • Installation and User Configuration
    • IPP v2.8
      • Release Notes
      • Installation and User Configuration
      • Manual Upgrade
    • IPP v2.7
      • Release Notes
      • Installation and User Configuration
    • IPP v2.6
      • Release Notes
      • Installation and User Configuration
      • Manual Upgrade
  • Sample Prep
    • QC and Sample Prep
      • DNA Initial QC 5.1.2
      • RNA Initial QC 5.1.2
      • Library Validation QC 5.1.2
  • Library Prep
    • AmpliSeq for Illumina
      • BRCA Panel
        • Library Preparation v1.1
        • Equalizer v1.1
        • Standard v1.1
      • Cancer HotSpot Panel v2
        • Library Preparation v1.1
        • Equalizer v1.1
        • Standard v1.1
      • Childhood Cancer Panel
        • DNA Library Prep v1.1
        • RNA Library Prep v1.1
        • Equalizer v1.1
        • Standard v1.1
      • Comprehensive Cancer Panel
        • Library Preparation v1.1
        • Equalizer v1.1
        • Standard v1.1
      • Comprehensive Panel v3
        • DNA Library Prep v1.1
        • RNA Library Prep v1.1
        • Equalizer v1.1
        • Standard v1.1
      • Custom DNA Panel
        • Library Preparation v1.1
        • Equalizer v1.1
        • Standard v1.1
      • Focus Panel
        • DNA Library Prep v1.1
        • RNA Library Prep v1.1
        • Equalizer v1.1
        • Standard v1.1
      • Immune Repertoire Panel
        • RNA Library Prep v1.1
        • Equalizer v1.1
        • Standard v1.1
      • Immune Response Panel
        • RNA Library Prep v1.1
        • Equalizer v1.1
        • Standard v1.1
      • Myeloid Panel
        • DNA Library Prep v1.1
        • RNA Library Prep v1.1
        • Equalizer v1.1
        • Standard v1.1
      • TCR beta-SR Panel
        • DNA Library Prep v1.1
        • RNA Library Prep v1.1
      • Transcriptome Human Gene Expression Panel
        • RNA Library Prep v1.1
        • Equalizer v1.1
        • Standard v1.1
    • Library Prep Validation
    • Nextera
      • Nextera Mate Pair v1.0
      • Nextera Rapid Capture Custom Enrichment v2.0
      • Nextera XT v2.0
    • Targeted Enrichment
      • Illumina DNA Prep with Enrichment (S) Tagmentation v1.2
      • Illumina RNA Prep with Enrichment (L) Tagmentation v1.1
    • TruSeq
      • TruSeq ChIP-Seq v1.0
      • TruSeq Custom Amplicon v1.0
      • TruSeq DNA Exome v2.0
      • TruSeq DNA PCR-Free v2.0
      • TruSeq Methyl Capture EPIC v2.0
      • TruSeq Nano DNA v1.0
      • TruSeq RNA Access v2.0
      • TruSeq RNA Exome v1.0
      • TruSeq Small RNA v1.0
      • TruSeq Stranded mRNA v2.0
    • TruSight
      • TruSight Oncology 500 ctDNA v1.1
      • TruSight Oncology 500 HT v1.1
      • TruSight Oncology 500 v1.1
      • TruSight Tumor 170 v2.0
    • Other DNA Protocols
      • Illumina DNA PCR-Free Library Prep Manual v1.1
      • Illumina DNA Prep (M) Tagmentation v1.0
    • Other RNA Protocols
      • Illumina Stranded mRNA Prep Ligation 1.1
      • Illumina Stranded Total RNA Prep Ligation with Ribo-Zero Plus v1.1
  • iLASS & Infinium Arrays
    • iLASS
      • iLASS Infinium Genotyping v1.1
        • iLASS Infinium Batch DNA v1.1
        • iLASS Infinium Genotyping Assay v1.1
        • iLASS Infinium Genotyping with PGx Assay v1.1
      • iLASS Infinium Genotyping v1.0
        • iLASS Infinium Genotyping Assay v1.0
        • iLASS Infinium Genotyping with PGx Assay v1.0
    • Infinium Arrays
      • Infinium HD Methylation Assay Manual v1.2
      • Infinium HTS Assay Manual v1.2
      • Infinium LCG Assay Manual v1.2
      • Infinium XT Assay Manual v1.2
      • GenomeStudio v1.0
  • Applications
    • IGA
      • IGA v2.1
        • IGA Library Prep Automated v2.1
        • IGA NovaSeq Sequencing v2.1
    • Viral Pathogen Protocols
      • CDC COVID-19 RT-PCR
        • Sort Specimens to Extraction v1.1
        • Qiagen QIAamp DSP Viral RNA Mini Kit v1.1
        • Qiagen EZ1 Advanced XL v1.1
        • Roche MagNA Pure LC v1.1
        • Roche MagNA Pure Compact v1.1
        • Roche MagNA Pure 96 v1.1
        • bioMerieux NucliSENS easyMAG Instrument v1.1
        • bioMerieux EMAG Instrument v1.1
        • Real-Time RT-PCR Prep v1.1
      • Illumina COVIDSeq v1.6
      • Respiratory Virus Panel v1.0
  • Instruments & Integrations
    • Compatibility
    • Integration Properties
      • Integration Properties Details
    • Clarity LIMS Product Analytics
      • Supported Workflows
      • Workflow Customization
      • Clarity LIMS Product Analytics v1.4.0
        • Configuration
      • Clarity LIMS Product Analytics v1.3.1
        • Configuration
      • Clarity LIMS Product Analytics v1.3.0
        • Configuration
      • Clarity LIMS Product Analytics v1.2.0
        • Configuration
    • Illumina Run Manager
      • Illumina Run Manager v1.0.0
        • Installation and User Interaction
    • iScan
      • iScan System
      • iScan v1.2.0
        • Release Notes
        • BeadChip Accessioning, Imaging, and Analysis
      • iScan v1.1.0
        • Release Notes
        • BeadChip Accessioning, Imaging, and Analysis
      • iScan System v1.0
    • iSeq 100 Run Setup v1.0
    • MiniSeq v1.0
    • MiSeq
      • MiSeq v8.3.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
      • MiSeq v8.2.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
        • Manual Upgrade
    • MiSeq i100 (On-Prem)
      • MiSeq i100 On-Prem v1.0.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
    • MiSeq i100 (Hosted)
      • MiSeq i100 v1.0.0
        • Release Notes
        • Configuration
        • User Interaction, Validation and Troubleshooting
    • MiSeqDx
      • MiSeqDx Sample Sheet Generation (v1.11.0 and later)
      • MiSeqDx v1.11.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
      • MiSeqDx v1.10.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
        • Sample Sheet Generation
        • Manual Upgrade
    • Next Generation Sequencing Package
      • Release Notes
        • NGS Extensions v5.25.0
        • NGS Extensions v5.24.0
        • NGS Extensions v5.23.0
      • Accession Kit Lots
      • Auto-Placement of Reagent Indexes
      • Compute Replicate Average
      • Copy UDFs
      • Initialize Artifact UDFs
      • Label Non-Labeled Outputs
      • Linear Regression Calculation
      • Normalization Buffer Volumes
      • Process Summary Report
      • Routing Script
      • Set UDF
      • Validate Complete Plate
      • Validate Sample Count
      • Validate Unique Indexes
    • NextSeq 500/550
      • NextSeq 500/550 v2.5.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
        • Manual Upgrade
      • NextSeq 500/550 v2.4.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
      • NextSeq 500/550 v2.3.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
    • NextSeq 1000/2000 (Hosted)
      • NextSeq 1000/2000 v2.5.1
        • Release Notes
      • NextSeq 1000/2000 v2.5.0
        • Release Notes
        • Configuration
        • User Interaction, Validation and Troubleshooting
        • Manual Upgrade
      • NextSeq 1000/2000 v2.4.0
        • Release Notes
        • Configuration
        • User Interaction, Validation and Troubleshooting
    • NextSeq 1000/2000 (On-Prem)
      • NextSeq 1000/2000 On-Prem v1.0.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
    • NovaSeq 6000 (API-based)
      • NovaSeq 6000 API-based v3.7.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
      • NovaSeq 6000 API-based v3.6.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
        • Manual Upgrade
    • NovaSeq 6000 (File-based)
      • NovaSeq 6000 File-based v2.6.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
      • NovaSeq 6000 File-based v2.5.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
    • NovaSeq 6000Dx (API-based)
      • NovaSeq 6000Dx API-based v1.3.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
      • NovaSeq 6000Dx API-based v1.2.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
    • NovaSeq X Series (Hosted)
      • NovaSeq X Series v1.3.0
        • Release Notes
        • Configuration
        • Manual Upgrade
      • NovaSeq X Series v1.2.1
        • Release Notes
      • NovaSeq X Series v1.2.0
        • Release Notes
        • Configuration
        • User Interaction, Validation and Troubleshooting
        • Manual Upgrade
      • NovaSeq X Series v1.1.0
        • Release Notes
        • Configuration
        • User Interaction, Validation and Troubleshooting
    • NovaSeq X Series (On-Prem)
      • NovaSeq X Series On-Prem v1.0.0
        • Release Notes
        • Installation
        • Configuration
        • User Interaction, Validation and Troubleshooting
    • References
      • Configure Multiple Identical netPathPrefixSearch Values
      • Configure Support for Samples Having Duplicate Names with Different Indexes
      • Illumina Instrument Sample Sheets
      • Terminology
  • Integration Toolkits
    • Lab Instrument Toolkit
      • Template File Generator
        • Creating Template Files
        • Template File Contents
        • Template File Generator Troubleshooting
      • Add Blank Lines
      • Convert CSV to Excel
      • Parse CSV
      • Name Matching XML Parser
      • Sample Placement Helper
    • Lab Logic Toolkit
      • Working with Lab Logic Toolkit
        • Data Collection Entities
        • Failing a Script
        • Mapping Field Types
        • Non-UDF/Custom Field Properties
        • Setting QC Flags
        • Setting Next Actions
        • Specifying Custom Fields
        • Working with Submitted Samples
        • Working with Containers
      • Lab Logic Toolkit Script Examples
        • Comparing Stop/Start Dates and Times with LLTK
      • Lab Logic Toolkit FAQ
  • Known Issues
    • Integration
      • Sample Sheet Generation Issue and CLPA Issues When Samples Have Been Assigned QC Flag Prior to Entering Steps
  • Security Bulletin
    • Investigation of OpenSSH vulnerability with Clarity LIMS
  • Resources
    • Third Party Software Information
  • Others
    • Revision History
Powered by GitBook
On this page
  • The Script
  • Workflow and Configuration
  • Script Parameters
  • Script Usage
  • Parameter Details
  • Additional Information

Was this helpful?

Export as PDF
  1. Integration Toolkits
  2. Lab Instrument Toolkit

Parse CSV

Available from: Clarity LIMS v2.0.5

-haltOnMissingSample option and support for header section values (e.g., containerName) are introduced in NGS v5.4.0.

Data might sometimes need to be parsed from an instrument result file (CSV, TSV, or other character-separated format) into Clarity LIMS, for the purposes of QC.

For example, suppose that a 96 well plate is run on a Caliper GX. The instrument produces a result file, which the user imports into Clarity LIMS. The per-sample data are parsed and stored for a range of capabilities, such as QC threshold checking, searching, and visibility in the Clarity LIMS interface.

The parseCSV script allows for the data for each well to be parsed into fields on either derived samples or result files (measurement records) that map directly to the derived samples being measured.

If the instrument result file contains data that applies to the batch of derived samples being measured, this data are stored in fields on the step.

The Script

The parseCSV script automates parsing a separated-value file, configurable but typically comma- or tab-separated, into the LIMS.

  1. Data lines in the file are matched to the corresponding sample in the LIMS using well placement information.

  2. A line that references well A1 of container Plate123 will have its parsed data mapped to the sample placed in well position A:1 of container Plate123 in the LIMS.

  3. Values from the file are mapped to fields (known as UDFs in the API) in Clarity LIMS based on the automation configuration for the script.

Workflow and Configuration

  • Configure the step to invoke the script manually via a button in Record Details screen.

  • Before pressing the button that invokes the script, upload a shared result file to be parsed.

  • Configure the automation command line to match the destination fields configured in Clarity LIMS.

  • Create a field for each column that will be brought into the LIMS. Field names must not contain the separator used for the automation parameter string, "::".

  • When using NGS v5.0 or later, fields can be configured for the step, input samples, output samples, or output result files. Versions before this release support only output result files.

  • Input result files are not supported.

Script Parameters

Parameter

Description

-u {user}

LIMS username (Required)

-p {password}

LIMS password (Required)

-i {URI}

LIMS process URI (Required)

-inputFile {result file}

Instrument result file to be parsed (Required)

-log {log file name}

Log file name (Required)

-containerName {container name}

Name of column header for container name

-wellPosition {well position}

Name of column header for well position

-sampleLocation {sample location}

Name of column header for <container_name>_<well>

-measurementUDFMap {measurement UDF map}

-partialMatchUDFMap {partial match UDF map}

-processUDFMap {process UDF map}

-headerRow {header row}

Numeric index of CSV header row, starting from one (default 1)

-separator {separator}

File separator; comma used by default if not otherwise specified (default comma)

-matchOutput {boolean}

-setOutput {boolean}

-relaxed {boolean}

-haltOnMissingSample {boolean}

Association Strategy

The association strategy describes how information in the file is mapped to samples in the LIMS.

When running this script, there are two association strategies you can implement. Which strategy you choose is determined by the contents of the file that will be parsed. Both strategies rely on sample placement information (well and container name) to perform the mapping to the LIMS.

  • Strategy 1: Provide the -containerName and -wellPosition parameters to the script. Use this strategy when the well and container information are found in separate columns of the file, eg "Plate123" in column "Plate Name" and "A1" in column "Well Label"

  • Strategy 2: Provide the -sampleLocation parameter to the script. Use this strategy when the placement information is all found in the same column, in the following format: <container_name>_< well >_<free text>, eg "Plate123_A1_control" in column "Sample ID"

Header Section Parsing (NGS v5.4 and later)

For the association strategy provided, if matching headers are not found in the file at the provided header index, the script will then search the lines of the file that appear prior to this index (the header section) for a match.

For example, when using association strategy 1 and providing -containerName and -wellPosition, if the file contains information for only a single container the container name may only appear one time in a header section. This may look something like this for a comma-separated file: "ContainerID, plate123". With -containerName provided as "ContainerID" the script will locate the adjacent value as the one to be used as the value of the container name for the entire file and interpret the well positions as being within this container.

Mapping Parameters

Mapping parameters (measurementUDFMap, partialMatchUDFMap, and processUDFMap) determine which information is mapped from the file to fields in the LIMS.

The structure in which to provide these parameters is as follows, where the <Header Name> is the name of the data column or header section row in the file:

<UDF Name>::<Header Name>

File Separator

While the most common file formats are *.csv (comma-separated) and *.tsv (tab-separated), the script may be configured to use any separator.

To use a comma or tab as the separator, provide these using the -separator parameter as "comma" or "tab" as they require additional handling by the script.

Boolean Parameters

The script supports several boolean parameters. Boolean parameter values must be provided in quotes, eg "true".

Script Usage

Example 1

This example uses matching Strategy 1 for a comma-separated file and maps two columns, "Region[100–1000] Conc. (ng/ul)" and "Region[100–1000] Size at Maximum [BP]", to output resultfile fields "Concentration" and "Size (bp)" in the LIMS, respectively:

bash -c "/opt/gls/clarity/bin/java
-jar \/opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar \
-i {processURI:v2:http} \
-u {username} \
-p {password} \
script:parseCSV \
-inputFile {compoundOutputFileLuid0} \
-log {compoundOutputFileLuid1} \
-headerRow '1' \
-separator 'comma' \
-containerName 'Plate Name' \
-wellPosition 'Well Label' \
-measurementUDFMap 'Concentration::Region[100–1000] Conc. (ng/ul)' \
-measurementUDFMap 'Size (bp)::Region[100–1000] Size at Maximum [BP]'"

Example 2

This example uses matching Strategy 2 for a tab-separated file, running in relaxed mode. It maps a column to an input sample field, using that input sample placement information, and maps a header section row to a protocol step field:

bash -c "/opt/gls/clarity/bin/java -jar \/opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar \
-i {processURI:v2:http} \
-u {username} \
-p {password} \
script:parseCSV \
-inputFile {compoundOutputFileLuid0} \
-log {compoundOutputFileLuid1} \
-headerRow '1' \
-separator 'tab' \
-sampleLocation 'Sample ID' \
-measurementUDFMap 'Concentration::Region[100–1000] Conc. (ng/ul)' \
-processUDFMap 'Run date::Run Date'
-matchOutput 'false'
-setOutput 'false'
-relaxed 'true'"

To view an out-of-the-box example included in the NGS package, review the configuration of the NanoDrop QC protocol steps included in the Initial DNA and RNA QC protocols.

Parameter Details

measurementUDFMap

This performs a 1:1 parsing of column information from the file to individual sample fields in the LIMS. The column names must match exactly. The exact destination (input/output sample or result file fields) is controlled through other script options.

partialMatchUDFMap

This allows customization of the column names that appear in the file by only matching on the first part of the column name, eg a partial match of "Sample" will match to a column customized to "Sample (internal ID)." Other than providing this flexibility, this parameter functions the same as measurementUDFMap.

If two columns are found that begin with the partial match provided, the script will log an error and stop execution.

processUDFMap

The process UDF option is provided to parse per-run information into protocol step fields in the LIMS. When provided, the script will search for a match in the header section and the data column headers of the file.

In the following example file:

  • The first two lines (beginning with OPERATOR and WORKFLOW) represent a header section with information for the batch of derived samples.

  • The third line (S_PLATE_ID) is the data section header (header row).

  • The lines make up the data section, which contains data for each derived sample.

How it Works

  • If there is a matching header in both the header section and column headers, the value from the header section will be used.

  • If no matching header is found and the script isn't running in relaxed mode, the script will log an error and stop execution.

  • When a match is found only among the column headers, validation is done to ensure all the values in that column are equal (because they will be mapped to a single destination field). If not all of the values are the same, a warning will be logged listing the distinct values and the field in the LIMS will not be updated.

matchOutput Mode

This parameter is provided as a boolean true/false value (default is false). It toggles whether information from the file is matched to the LIMS by comparing it to the placement of the protocol step inputs or protocol step outputs.

  • If set to False: The script uses the placement information of the inputs.

  • If set to True: The script uses the placement information of protocol step outputs.

setOutput Mode

This parameter is provided as a boolean true/false value (default is true). It toggles whether per-sample information is mapped to fields on the protocol step inputs or outputs.

  • If set to True: The script updates the protocol step outputs.

  • If set to False: The script updates field information on the protocol step inputs.

Input samples, output samples, and output result files are supported. The script expects either output samples or output result files, not both.

The script will log an error and stop execution if there is more than one kind of per-input output configured for the protocol step.

relaxed Mode

This parameter is provided as a boolean true/false value (default is false) to toggle relaxed mode.

  • If set to False: The script considers all provided header mappings to be mandatory headers and throws an exception if anything cannot be found in the file.

  • If set to True: In relaxed mode, the script will log a warning if a header cannot be found in the file, and will continue execution.

haltOnMissingSample mode (NGS v5.4 and later)

This parameter is provided as a boolean true/false value (default is true) to toggle halt on missing sample mode.

  • If set to False: The script will warn but continue execution when placement information for a line in the file cannot be determined. This mode can be used to handle, for example, ladder entries or footer sections, where the lines in the file will not contain valid sample information for the parser to use.

  • If set to True: The script will log an error and stop execution when a line in the file is encountered where it cannot determine the placement information for a sample. This mode allows strict matching of all contents.

Additional Information

Other scripts you may find useful are as follows.

PreviousConvert CSV to ExcelNextName Matching XML Parser

Last updated 4 months ago

Was this helpful?

Mapping of CSV columns to fields in the LIMS. See .

Mapping of CSV columns to fields in the LIMS (partial match). See .

Mapping of CSV headers or columns to protocol step fields in the LIMS. See .

Match by input or output placement (default false). See .

Control which artifacts to parse information into (default true). See .

Control whether headers are optional or mandatory (default false). See .

Control whether the script halts execution or warns when the Container Name and Well Position cannot be determined on any line (default true) (NGS v5.4 and later). See .

At least one mapping parameter must be provided to map data from the file to the LIMS. The details of how each of these parameters affects the behavior of the script is described in the section.

Sample Input Sheet Generator
Name Matching XML Parser
Parameter Details
Parameter Details
Parameter Details
Parameter Details
Parameter Details
Parameter Details
Parameter Details
Parameter Details