• Repurposing a Process to Upload Indexes

• Adding Users in Bulk

• Moving Reagent Kits & Lots to New Clarity LIMS Server

Compatibility: API version 2

Uploading indexes (reagent types) into BaseSpace Clarity LIMScan be done 3 different ways:

1. Manually adding the indexes one at a time using the Operations interface.

2. Uploading an XML data file using the config slicer.

3. Using the API directly.

There is no "out of the box" method to quickly and easily upload a CSV file of many indexes to the LIMS. Adding the indexes one at a time is not quick, and using the config slicer requires admin privileges, knowledge of the correct XML file format, and the command line.

Solution

This example enables Clarity LIMS lab users to upload a CSV file containing new indexes and, through an EPP trigger, instantly create the indexes in Clarity LIMS. This example provides the provides the functionality included with the config slice, including preventing the indexes from being created if there is already an index with the same name in the system as well as ensuring the sequence only contains a valid nucleotide sequence (composed of A,G,T and C).

Protocol Step Configuration

In this example, a protocol step is repurposed as a mechanism to upload indexes. The protocol is configured to not produce any analyte outputs, and a single shared output file is used as the placeholder to upload the CSV file containing the indexes. The step is configured to permit use of a control sample, which allows the user to begin the step without any sample inputs.

Parameters

The script accepts the following parameters:

An example of the full syntax to invoke the script is as follows:

User Interaction

The lab scientist begins the step with only a control in the ice bucket. The user uploads the CSV file to the placeholder on the record details screen, and pushes the button to trigger the script.

About the Code

The main method in the script is importIndexes(). This method first calls the function downloadfile() which finds the location of the CSV file on the Clarity LIMS server and reads the content into memory.

The script then generates an XML payload for each index, checks the nucleotide sequence against the acceptable characters, and POSTs to the /api/v2/reagenttypes endpoint. If an index with the same name is already present in the LIMS, the log will be appended with the exception.

This example script produces a log which tracks which indexes were not added to the system as well as the reason. The log displays how many indexes were successfully added to the system and the total number of indexes included in the CSV file.

Assumptions and Notes

• You are running a version of Python that is supported by Clarity LIMS, as documented in the Clarity LIMS Technical Requirements.

• The attached files are placed on the LIMS server, in the /opt/gls/clarity/customextensions folder.

• You will need to update the HOSTNAME global variable such that it points to your Clarity LIMS server.

Attachments

example_Indexes.csv:

addindexescsv.py:

Compatibility: API version 2

The Clarity LIMSweb interface provides an example Sample Import Excel file which can be manually uploaded to submit new samples to a selected project within the LIMS.

This application example shows how the Sample Import Sheet can be uploaded programatically with just one change to its format.

Solution

This example uses the same Sample Sheet with an additional column 'Project/Name'. The script processes the Sample Sheet, creates the samples and adds the samples to their respective projects.

This script leverages a python module xlrd, which is not included in the standard python library. It is used to extract data from .xls and .xlsx excel files.

Parameters

The script accepts the following parameters:

An example of the full syntax to invoke the script is as follows:

About the code

parseFile

This method carries out several operations:

• Opens the excel file and reads the text

• Stores the column headers in a dictionary variable called COLS

• Stores the row data accessibly in an array variable called ROWS

createProject

This method in turn carries out several operations:

• For each project name the script encounters it searches the LIMS to identify if a project with this name has already been created.

• If the project does not exist, the script will create the project. This example script is easily modifiable, however as written:

processRows

This method prepares the data needed to create a sample in LIMS:

• Assembles the UDF values, the project ID and container ID.

• For each non-tube container the script encounters it searches the LIMS to identify if a container with this name already exists.

• If the container does not exist the script will create the container.

The script contains additional supporting methods to generate XML which is POSTED to the API.

Assumptions & notes

• The UDFs in the Sample Sheet header have been configured in LIMS prior to submission.

• The following column headers are required: Sample/Name, Project/Name and any sample-level UDFs that are mandatory within your system.

• The script need not be run on the Clarity server, however it must have a connection to the Clarity LIMS API.

Attachments

_auth_tokens.py:

ClaritySampleSheetprojects.xlsx:

SampleSheetImporter.py:

It is sometimes necessary to assign a sample to a new workflow from within another workflow.

You can do this in the BaseSpace Clarity LIMS Operations Interface, by manually adding the sample to the desired workflow. However, it is also possible to perform this action through the API.

Solution

This example shows how to use the API to automate the addition of samples to a specified workflow, based on a UDF value.

• The script can be run off any protocol step whose underlying process is configured with Analyte inputs and a single per-input ResultFile output.

• The process may have any number of per-all-input ResultFile outputs, as they will be ignored by the script.

• A Result File UDF, named Validate, will control which samples will be added to the specified workflow.

Parameters

The script accepts the following parameters:

Step 1: Create Result File UDF

Before the example script can be used, first create the ResultFile's Validate UDF in the Operations Interface. This is a single-line text UDF with preset values of 'Yes' and 'No'.

Step 2: Create and Configure Process Type

Also in the Operations Interface, create a new process type named Cookbook Workflow Addition.

This process type must:

• Have Analyte inputs.

• Have a single per-input ResultFile output.

• Apply the Validate UDF on its ResultFile outputs.

Step 3: Configure an EPP call on this process type as follows:

Step 4: Modify the file paths to suit your server's Groovy installation.

Step 3: Create Protocol

Once the process type is created, in the Clarity LIMS Web Interface, create a protocol named Cookbook Workflow Addition Protocol.

This protocol should have one the protocol step - Cookbook Workflow Addition.

Step 6: Configure EPP

Configure the EPP script to automatically initiate at the end of the Cookbook Workflow Addition step:

Step 7: Create workflows

To finish configuration, create two workflows:

• Destination Workflow: THis workflow should contain the DNA Initial QC protocol only.

• Sending Workflow: This workflow should contain the new Cookbook Workflow Addition Protocol.

About the code

Once the script has processed the input parameters and ensured that all the required information is available, we can start processing the samples to determine if they should be assigned to the new workflow.

1. To begin, we retrieve the process from the API. This gives us access to the input-output maps of the process. These will be used to determine which ResultFiles we will examine.

2. Next, we retrieve the protocol step action list. This contains a list of the input analytes' URIs and their next steps.

3. We then search this list for and collect all analyte URIs whose next action has been set to Mark as protocol complete.

User Interaction

1. Assuming samples have been placed in the Switching Workflow, the user proceeds as normal through the protocol step.

2. In the Record Details screen, the user enters Validate values in the ResultFile UDFs.

3. The user then proceeds to the Assign Next Steps screen, provides a variety of Next Steps, and completes the protocol step.

Assumptions and Notes

• The attached file is placed on the Clarity LIMS server, in the /opt/gls/clarity/customextensions folder.

• GLSRestApiUtils.groovy is placed in the Groovy lib folder.

• The required configuration has been set up, as described in Configuration.

Attachments

SwitchingWorkflows.groovy:

An easy way to get sample data into BaseSpace Clarity LIMS is to import a Microsoft® Excel® sample spreadsheet. However, manually configuring the spreadsheet can be time-consuming and error-prone.

Use this Python application example to generate a custom Excel sample submission spreadsheet that meets the specific needs of your lab.

Solution

Whereas config slicer is a useful tool for moving configuration data from one server to another and, strictly speaking, reagent lot data is not considered configuration, there is a need for an easy way to transfer reagent lot data to a new Clarity LIMSserver.

Solution

The preventative solution is to not create production reagent kits on the development server. However, if you're reading this it might be too late for that.

This example demonstrates how to use the Clarity LIMS API to extract reagent kit and reagent lot data from one server, transfer the data as an XML file and create the equivalent reagent kits and/or lots on the new server.

When Clarity LIMS is replacing an existing LIMS, or if the LDAP integration is not being utilized, a list of current LIMS users and collaborators often already exists in a file.

In this scenario, it is useful to use this file to create users in Clarity LIMS. This topic provides an example script and outlines a strategy for the bulk import of users from a simple CSV file.

Solution

The attached script parses a CSV file containing user information. Based on the columns of data present in the file, the script then creates users and their associated labs in Clarity LIMS.

Parameters

Since this operation is independent of workflow and sample processing, the parameters used to invoke the script differ from those typically used:

An example of the full syntax to invoke the script is as follows:

File format

The format of the file is very flexible. The order of the columns is not relevant.

If the names of your columns do not match the column names in the example file, modify the script so that the column names match yours.

Attached to this topic, you'll find a CSV file containing test data that illustrates the format further. The structure of this file is shown below.

Notice that the first two rows do not contain data of interest, and so the script ignores them. This is controlled by line 38 of the script, which specifies the location of the header row:

About the Code

The main method of interest is importData(). After parsing the file into data structures (COLS and DATA), the data is processed one line at a time using the following pseudocode:

To reduce the number of queries the script makes back to the server, each time a new lab is created it is added to the cache of existing labs that is stored in the LABS dictionary.

Finally, if you set the DEBUG variable to true, the script will stop processing the file after the first successful creation of a user. This is useful as it allows you to test your script using just one line of the CSV file at a time.

Assumptions and Notes

• Both of the attached *.py script files are placed on the Clarity LIMS server, in the /opt/gls/clarity/customextensions folder.

• The users will be created with the same hard-coded password (abcd1234). It is possible to have the script create a unique password for each user. If this is required, consider having the script e-mail the user with a 'welcome' e-mail outlining their username and password.

• The users will be created with permissions to access the Collaborations Interface only. This can be modified as required.

Attachments

user_List_Test.csv:

userImport.py: