If necessary, the Clarity LIMS Support team can provide a backup of your Clarity LIMS data. The data are contained in an encrypted file, which can be downloaded from a secure SFTP server.

Prerequisites

To receive the backup data file, provide the Clarity LIMS Support team with a GNU Privacy Guard (GPG) public key.

For instructions on generating a GPG public key, see the following documentation:

• For Microsoft® Windows®, see .

• For Linux® or Mac®, see .

Receiving and Decrypting your Backup Data

1. After the Clarity LIMS Support team has received the GPG public key, they do the following actions:

   • Create a backup file encrypted with the key.

   • Place the backup file on the LIMS SFTP server at sftp.clarity-lims.com.

After initial user training, but before the lab starts to use Clarity LIMS in production, a database cleanup is recommended.

This process removes any projects, samples, workflows, protocols, steps, and other artifacts from the system that were created during training but are not needed for production.

Contact the Clarity LIMS Support team to schedule a time for this cleanup procedure.

Preparation

This section provides information and instructions to support Clarity LIMS and administration tasks. Topics covered include

• Database Cleanup

• LDAP Integration

If you require assistance with Clarity LIMS administration, contact the Illumina Support Team.

This section explains how to use the LDAP Checker tool a script (ldap-checker.jar) that checks and reports on an LDAP configuration. Instructions for use are also provided in the README.txt file that accompanies the tool.

The ldap-checker script is included with the Clarity LIMS installation and is available at the following location:

/opt/gls/clarity/tools/ldap-checker

The ldap-checker script performs numerous checks of the LDAP configuration and reports on any incorrect items found.

Script Properties

Point the script to one or more files containing (at a minimum) the database connection properties. Alternatively, set these properties from the command line.

The script loads properties from the following sources and in the following order:

1. Any JDBC properties files specified with -f (see the table for options).

   If multiple properties files specify the same property, the last file is used.

2. Any Java system properties specified on the command line using

   Properties specified on the command line are only checked if they do not appear in the properties files.

After the script has the basic database connection properties, it loads further settings from the corresponding Clarity LIMS database.

The following JDBC properties are required:

• jdbc.driverClassName

• jdbc.url

• jdbc.username

• jdbc.password

Options:

Run the Script

1. Change to the directory containing ldap-checker tool:

2. Run the script. To specify a properties file, use the following example:

Example Properties File

The tool includes an example database.properties file. This example shows a properties file that is specified with the -f option.

The following options are available:

• Edit this file and use it.

• Provide properties on the command line, using: -D.

  For example:

Examples

Example 1: Using SSL

Specify and provide the path to the keystore:

Example 2: Checking Users

To check a set of specific users (even those that have not been provisioned), use the following script:

Overriding Properties

To override properties that are typically loaded from the properties table, use command-line system properties or one or more properties files.

Using system property ( -D options must be specified before the -jar option):

Using multiple properties files:

• In this example, Custom-ldap.properties might resemble the following:

Clarity LIMS provides Audit Trail, a robust data-tracking system that allows for tracking of the following:

• All user activity (ie, who did what, and when).

• Every action that is written to the database.

Audit Trail has two capture systems, Event Log and Detail Log.

You can configure an alias for the short and long, singular, and plural forms of the term Project, as displayed in the Clarity LIMS interface.

Configuration Tool and Properties

Renaming is achieved by configuring the following properties, using the omxprops-ConfigTool.jar tool at:

Copy

/opt/gls/clarity/tools/propertytool 

Properties

• clarity.alias.project.short.singular - The short term for "Project"

• clarity.alias.project.short.plural - The short term for "Projects"

• clarity.alias.project.full.singular - The full term for "Project"

• clarity.alias.project.full.plural - The full term for "Projects"

Rules for alias

• clarity.alias.project.short.singular—Maximum of eight characters.

• clarity.alias.project.short.plural—Maximum of nine characters.

• If multiple words are used, capitalize each word (eg, Test Requests).

If running Config Slicer v3.0.x against a configuration package or manifest file that was generated with a pre-3.0 version of Config Slicer, an error message displays.

Scenario 1: Out-of-Date Configuration Package

In this scenario the error message will resemble the following:

Copy

Package was generated with an older version of config slicer incompatible with the current version (3.0.2).
Please upgrade the configuration package (java -jar upgrade-config-slice.jar 'configuration_package.xml'),
verify the results, and try again. 

SOLUTION:

Upgrade the configuration package file by running upgrade-config-slice.jar against it.

• This jar file should be located in the same directory as Config Slicer (/tools/config-slicer).

• The upgrade script will save an upgraded copy of the configuration package file (to the same directory), which can be inspected and imported.

Example:

Copy

To upgrade a manifest file at the same time, simply add it to the script as a second argument:

Copy

Scenario 2: Out-of-Date Manifest File with no Corresponding Configuration Package

In this scenario the error message resembles the following:

Copy

SOLUTION:

This upgrade process is a little more involved:

1. Extract the list of process types from the manifest file
.

2. Format them as parameters to Config Slicer's custom manifest generation for process types.

   For example, assuming we extracted the process types Process Type 1 and Process Type 2 in step 1, the following command would be used:

   Copy

Create a Configuration Package File for Import to Another System (or for Backup Purposes)

Use a configuration package file to copy a configuration set from one server to another or to back up a particular working configuration at a particular time.

Required steps:

1. Create a configuration manifest file.

If there is an API version mismatch, Config Slicer will log a message at the beginning of an import:

Copy

Generally, this message is not a cause for concern.

However, if there are warnings about configuration differences during the import, changes that have been made to the API in between the package version and the server version may be responsible.

In addition, if the package being imported is from v4.x, a log message may appear at the beginning of an import:

Detected package is from a pre-5.0 Clarity system. The package will be upgraded to match new configuration requirements.

In this case, the package is upgraded and the resulting configuration is output to a folder in the same location as the package being imported. This upgraded configuration package is the v5.x representation of the v4.x configuration and can be used directly to troubleshoot error occurring on import. The updated packaged can also be imported directly.

Config Slicer Version Updates

The following changes have been included in the latest release of Config Slicer (v 3.0.51):

General changes:

• The entity summary is now shown after the individual differences have written to the log file, as opposed to before these differences.

• Support was dropped for the following process type attributes which are no longer used:

  • SupportsExternalProgram,

Note: If upgrading the package fails, import/validation will fail. In general, this will reflect a mistake in the configuration package being imported.

Changes in support of slicing from 4.x to 5.0:

The following changes were specifically added to support slicing between Clarity LIMS 4.x and BaseSpace Clarity LIMS 5.0. All of these changes will be saved into a new configuration package that will be written to the same location as the package being sliced:

• Backwards compatibility for Protocol Step setup configuration.

• Support for updates of parent entities after both the parent and child entities have been imported.

  • This change is required for updating the defaultProcessTemplate and step-fields step properties on Master Steps. Both require that the Master Step (ProcessType), ProcessTemplate, and Master Step Custom Fields (ProcessType UDFs) exist before they can be set on the Master Step.

After slicing, it is recommended to examine the configuration closely for any QC Protocol Steps that previously shared a Master Step with nonQC Protocol Steps. This setting may not transfer as expected and there is potential for misconfigured Protocol Steps. In particular, if the Master Step produces measurements, and even just one child step of a Master Step has qcProtocolStep=true, then the Master Step will get qcProtocolStep=true set on it. Thus, all other steps that use that Master Step have qcProtocolStep=true, whether or not it was set before.

• Support for setting the default container on Steps and Master Steps in such a way that all behaviour is maintained from 4.2

  • If every child step of a Master Step has the default container that was defined as a permitted container on the master (through the 'OutputContainerType' process-type-attribute) then the default will be added as a permitted container on the master step

  • If any child does not have the default container from the Master Step, then the default is removed from the Master and each child that had the container is updated so that the default is the first permitted container (and hence default)

All Clarity LIMS installations include the installation of a separate component known as an Automation Worker (AW) node (formerly known as Automated Informatics (AI) node).

When writing automation-based triggers, the code invoked by an automation runs on the AW node.

While the AW node is a critical component, it typically does not require much attention. However, there are many other options that must be considered. These options include how many AW nodes a Clarity LIMS system uses, and where they are placed. This section discusses some of these options.

Introduction

Clarity LIMS creates various log files to help with the resolution of issues. During support request investigation, the Support team may ask for the following types of log files:

The Config Slicer tool is used to move small incremental configuration changes, contained in a configuration set, between Clarity LIMS systems. For example, it moves changes between a test system and a production system.

This configuration tool provides granular export/import functionality that allows the management of configurations that support experimental workflows.

Use this tool to back up, copy, deploy, and restore configuration sets. By making small incremental changes, make sure that the modifications made to the production system are minimal.

Review the following key concepts:

• Configuration set—This item may be created by the Illumina Support team or by the customer. It comprises the items (know as entities) that are added to a Clarity LIMS system to allow for customization for a particular scientific experiment or workflow. The Illumina NGS Extensions Package is a good example. See

Use the following steps to help troubleshoot the installed Automation Worker framework. A flowchart is provided as a reference.

Flowchart Reference

Turning Container Name Uniqueness ON

This constraint is set at the database level. Container Uniqueness can be turned ON by using the migrator tool, and running the optional migration step called "ContainerNameUniqueConstraints". This can be done by calling this command:

Notes:

• Of course, edit the migrator.properties file to ensure that the mode is set to "migrate" not just "validate".

Saving passwords in encrypted format is recommended. Use the omxprops to do this action.

To encrypt a password:

Use the following command:

# java -jar /opt/gls/clarity/tools/propertytool/omxprops-ConfigTool.jar encryptPassword <password>

This command returns a text string resembling the following example:

RqHL5XpY0NStVRjd+BngRQ==  

To set an encrypted password:

Set the password by enclosing the text string in the ENC() wrapper.

Consider the following examples:

• When setting an encrypted password in a configuration file:\

• When using the property tool to set an encrypted password:\

• Using ENC() is not needed when setting the password in Automation Worker:\

Command-line Options and Usage

Refer to Import/Export Mode and Guidelines and Semantics for Creating Manifest Files.

* The importAndOverwrite option lets Config Slicer update existing configuration, rather than create new configuration. This option is only available in LIMS 3.4.

Usage

Copy

EXPORT: Creating a configuration package file

Prerequisites

• Created and validate a configuration set on the source server.

• Have access to the Config Slicer tool and the libs subdirectory.

Step 1: Create configuration manifest file

Create Simple manifest file or Custom manifest file.

Simple manifest file:

1. On the source server, copy and paste following code to the command line. Edit the variables (version, server IP address, username, password, and manifest file name) to match those in your system.

   Copy

   A command with the variables filled in might look like this:

   Copy

2. This step produces a manifest file containing information about the entire system configuration. For best practice, copy this file and rename the copy in a way that reflects the configuration (we'll use newconfiguration.txt for this example). Use the copied file for the next steps.

A manifest file is used as an intermediary step to produce an XML configuration package file.

The manifest file is only relevant to the data that exists in the system at the time it is created. Discard it after creating the configuration package file or save the manifest file for historical auditing purposes. It can provide a record of a known working configuration set on a particular system.

Custom manifest file:

To create a manifest file for only specific workflows, protocols, or process types, follow the steps outlined previously, using -o custom instead of -o example.

• When using this operation provide additional parameters (-w, -pr, and/or -pt) specify the exact entities for which to create a manifest. For example:

  Copy

• Specifying every option is not required. It is also possible to specify more than one of each kind. For example, create a manifest file for a workflow with the following command:

  Copy

Step 2: Edit Manifest File

The next step is to edit the manifest file, removing unnecessary information and preserving only the custom configuration to import into the destination system.

Example 1: In this example, everything is deleted from the manifest file, except for the two new process types to export.

Copy

Example 2: In this example, the manifest file contains definitions for some new reagent types:

Copy

Step 3: Export to XML Configuration Package File

• Copy and paste the following code onto the command line. Edit the variables (version, server IP address, username, password, and manifest and package file names) as required.

  Copy

• An edited command might look like the following example:

  Copy

This step generates a data file in an XML format (newconfiguration.xml in our example) that is compliant with the Rapid Scripting API.

IMPORT: Installing a Configuration Package File

Prerequisites

• A configuration package file has been exported. This example uses a file named newconfiguration.xml.

• Access to the Config Slicer tool on the destination server has been granted.

• There are no in progress steps for any of the protocols that are going to be sliced in, otherwise the import of the protocol fails.

Step 1: Import Configuration Package File

• On the destination server, copy and paste the following code to the command line. Edit the variables (version, package file name, server IP address, username, and password) as required.

  Copy

• A command with the variables filled in might look like this:

  Copy

About duplicate entities

If any of the configuration entities that are about to import, exist in the destination system, Config Slicer either logs a warning or attempts to update them. It depends on the mode being run (see ).

If Clarity LIMS has maintained an internal record of deleted items, the previous information may also apply to deleted entities. This situation may occur if those entities have created outputs that currently still exist in the system.

• When running in import mode, entities that exist and are different from the version in the package have a warning and full diff logged.

• When running in importAndOverwrite mode, Config Slicer attempts to update entities that exist and are different from the version in the package.

  • In this scenario, back up configuration package containing copies of the updated entities before they were changed is saved to the directory where the configuration package is located. If that directory is not writable, the backup package is saved to the same directory as the log file.

To avoid changing existing configuration (which could possibly break historical data), another option is manually renaming the old entities. Add an extension or a prefix and continue with importing the new configuration package.

Step 2: Validate the import

Use the following methods to validate whether an import has completed successfully:

Check the Import Log:

For each specific type of configuration that is being imported (e.g. container types, process types, workflows), Config Slicer will log a set of messages. The messages look similar to the following examples:

Copy

• Before it begins to process a specific entity, the file logs how many entities were found. Any errors or warnings about this set of entities always appear between Found 4 $Entities and Summary of $Entities.

• Every entity that is found in the configuration package always appears in the summary, in one category or another. If a scenario occurs where this isn't true, or where the initial count of entities does not match the number in the summary, something has gone wrong and a bug report should be filed.

Validate with Config Slicer:

Running Config Slicer with the validate operation checks every entity in the package to see if it exists on the destination server. It reports results in a format similar to the log format shown previously.

Run the validate operation before or after importing:

• Before importing—checks if there could be any problems when importing a configuration from a package. This feature is its primary use as it makes sure that during import, "configuration exists in package but not on server" is not considered an error case.

• After importing—makes sure that the results are what was expected.

Example of validate output:

Copy

Check Configuration on the Destination Server:

The ultimate test of whether configuration has imported successfully is to check the configuration on the destination server itself. Make sure it looks and behaves as expected.

Configuration can be checked either via the Configuration screen in the Clarity LIMS user interface, or via the configuration endpoints in the API.

Guidelines and Semantics for Creating Manifest Files

Top-level entities

• To be included in a configuration package file, the top-level entities of the custom configuration set must be explicitly enumerated.

• Some of the top-level entities are discrete 'self-contained' units, and do not include other units (for example, container types, reagent types, artifact groups, and non-artifact/non-process type UDFs).

• Some top-level entities (process types, for example) automatically include other units (refer to for more information).

Non-Top-Level Entities

• Some entities are only included as part of other entities. For example, process templates, process type UDFs, and artifact UDFs are only included when included in a top-level process type. (The latter is a special case, given that the same artifact UDF can be used by multiple process types.)

Required Entities

• Some entities may be required by other entities. In these cases, make sure that these entities are exported/imported in the correct order. For example, because process types may require the existence of a container type, create the container type first.

• Required entities are not automatically included. If they do not exist in the destination system, explicitly include them in the manifest file. For example, suppose that a process type declares a particular container type as a default output plate. If that container type does not exist in the destination system, include that container type in the manifest file.

Import/Export Mode

Import modes affect the transactability of the tool, allowing it to make incremental changes if errors occur or provide an all-or-none option. For example, use the operation validate mode to determine if errors were encountered.

Best-Effort Mode

• This is the default import mode.

• This mode attempts to import as many units as possible. Any failures are logged, but the import operation is not interrupted.

  • For example, a failing container type import will not prevent other container types from being processed.

Strict Mode

• If this mode is used, the import operation is aborted if it encounters a failure.

  • For example, if there is an API version mismatch, the operation will abort and no further imports will be executed. Note that any changes already performed are not reverted.

• Use the -Strict option to enable this import mode.

Validate Mode

Use the validate operation (instead of import) to enable this mode.

This mode produces a report listing showing the following items:

• Entities that would be successfully imported because they do not exist on the target server.

• Entities that already exist on the target server and are identical.

• Entities that already exist on the target server but are different.

Validate mode can only detect a limited set of errors. For example, it can check if a particular piece of configuration already exists. If so, it checks if it is identical to the one included in the configuration package.

This information can help determine if the importAndOverwrite option is needed instead. For details, see .

Example of console output:

Copy

Example Mode

• Use this mode to generate a manifest file if the configuration you want to export is not tied to a specific set of workflows, protocols, or process types.

• Use the example operation (instead of import) to enable this mode.

Enforcing Unique Sample Names Within a Project

By default, Clarity LIMS allows duplicate sample names within the same project. If you would like to enforce sample name uniqueness within a project, you can do so.

Two scripts have been developed to support this requirement:

If you use, or would like to use, an LDAP server to consolidate directory services, it is possible to integrate LDAP with Clarity LIMS.

The Clarity LIMS LDAP solution allows for the following features:

• User name and password authentication against LDAP to govern access to Clarity LIMS.

• Ongoing unidirectional synchronization of user information (such as first name, last name, title, phone, fax, and email) from LDAP to Clarity LIMS. For example, if your telephone number is changed in the LDAP directory, the information is pushed down to Clarity LIMS, keeping contact information current.

• Automated unidirectional provisioning of user accounts from LDAP to Clarity LIMS. For example, adding a user to a particular group within the LDAP directory automatically results in a local account with LDAP authentication being added to Clarity LIMS.

Providing Information about your LDAP Implementation

Our Field Application Specialist (FAS) team meets with you to discuss the current LDAP implementation. In preparation for this meeting, collect the following information:

• The type of provisioning you would like to use to synchronize Clarity LIMS with LDAP (automatic or manual).

• A list of the LDAP attributes the current system uses to record the following user properties: first name, last name, title, phone number, fax number, and e‐mail address.

NOTE: When integrating Clarity LIMS with LDAP, the LIMS database and the LDAP directory remain as separate and distinct entities.

Supported LDAP Servers

Clarity LIMS is tested with the following LDAP servers:

• ApacheDS 1.5 and later

• Microsoft Active Directory (Windows Server 2003 or later)

• OpenLDAP 2.3.35 and later

Access and Changes

While user provisioning and authentication are handled with LDAP, a Clarity LIMS system administrator completes the following steps:

1. Determine the level of access that a user requires.

2. Modify the userʹs account within the LIMS to provide that access.

Once an LDAP integration with Clarity LIMS is established, all changes to user profiles must be made from the LDAP server.

Provisioning Users

Only automatic user provisioning is available.

With automatic user provisioning, Clarity LIMS users are created automatically by a provisioning tool that periodically synchronizes the LDAP server with the LIMS.

To make use of the LDAP directory services, Clarity LIMS maps to specific LDAP attributes within a defined schema.

However, the directory structure used can vary among installations. Our Field Applications Specialist (FAS) team work with you to complete the following items:

• Analyze a specific LDAP solution and directory organization or assist with the selection and initial configuration of an LDAP service.

• Discuss the user elements that will be synchronized between the LDAP service and Clarity LIMS systems.

• Configure LDAP to connect to your Clarity LIMS systems.

Caching User Authentication Results from your LDAP Server

User authentication is handled in the Clarity LIMS.

In previous versions of Clarity LIMS, a few customers reported slow response time for the REST API when using LDAP users for authentication. As of Clarity LIMS v5.2.x / v4.3.x, the REST API response time has improved by introducing a new feature that caches user authentication results through a new property (api.session.timeout).

To make use of the new feature, do the following actions:

• Make sure that api.session.timeout property is set.

• Include the HTTP Connection & Authorization request headers and session cookie in the HTTP request.

Setting the api.session.Timeout Property

Stored in the Clarity LIMS database table, the api.session.timeout property allows you to specify the period of time for which a user's session should persist, after they have been authenticated.

This property is set during installation or upgrade of the LIMS. The default value is 5 minutes. If necessary, update the value using the omxprops-ConfigTool.jar tool at the following location:

For example:

For this configuration to take effect, stop and restart Tomcat:

Including the HTTP Authorization Request Header and Session Cookie

To persist user authentication, the HTTP request must contain the following HTTP request headers:

• Request Header

  • Connection: Keep-Alive

  • Authorization: Basic <credentials>

The HTTP request headers are required for the initial request, and for any subsequent request to get a valid JSESSIONID. Additional scenarios are described in the following table.

To make sure that a valid authenticated session is provided if the cookie in the request has expired, also provide the following JSESSIONID cookie:

• Cookie

  • JSESSIONID=<a valid JSESSIONID from the initial request>

The following table lists the various combinations of HTTP Authorization request header and JSESSIONID cookie and their expected result. It assumes that the HTTP Connection request header is provided for all scenarios.