Managing Configurations with Config Slicer
Command-line Options and Usage
Refer to Import/Export Mode and Guidelines and Semantics for Creating Manifest Files.
| Option | Description |
|---|---|
-a,--apiuri <apiuri> | The BaseSpace Clarity LIMS REST API base URI (ends in "/api") (Either this or --server must be provided) |
-k,--package <package file> | File to be imported from or exported to (Required if operation is import, importAndOverwrite*, export, or validate). If file is not local a full path is required. |
-f,--force <force>* | Force update without prompt when running in importAndOverwrite mode (Optional) |
-m,--manifest <manifest> | Manifest file (Required if operation is export or example). If file is not local a full path is required. |
-o,--operation <operation> | The operation mode for the Config Slicer tool. Options are import, export, validate, example, importAndOverwrite, and custom (Required) |
-p,--password <password> | The BaseSpace Clarity LIMS REST API password, if encrypted, use "ENC(<encrypted-password>" (Required) |
-pr,--protocols <protocols> | The protocols to include in the custom manifest (Optional) |
-pt,--processTypes <processTypes> | The process types to include in the custom manifest (Optional) |
-s,--server <server> | The BaseSpace Clarity LIMS REST API server (either this or --apiuri must be provided) |
-S,--Strict | Strict mode for import (fail fast - default mode is best-effort) (Optional) |
-u,--username <username> | The BaseSpace Clarity LIMS REST API username (Required) |
-w,--workflows <workflows> | The workflows to include in the custom manifest (Optional) |
* The importAndOverwrite option lets Config Slicer update existing configuration, rather than create new configuration.
Usage
EXPORT: Creating a configuration package file
Prerequisites
Created and validate a configuration set on the source server.
Have access to the Config Slicer tool and the libs subdirectory.
Step 1: Create configuration manifest file
Create Simple manifest file or Custom manifest file.
Simple manifest file:
On the source server, copy and paste following code to the command line. Edit the variables (version, server IP address, username, password, and manifest file name) to match those in your system.
A command with the variables filled in might look like this:
This step produces a manifest file containing information about the entire system configuration. For best practice, copy this file and rename the copy in a way that reflects the configuration (we'll use newconfiguration.txt for this example). Use the copied file for the next steps.
A manifest file is used as an intermediary step to produce an XML configuration package file.
The manifest file is only relevant to the data that exists in the system at the time it is created. Discard it after creating the configuration package file or save the manifest file for historical auditing purposes. It can provide a record of a known working configuration set on a particular system.
Custom manifest file:
To create a manifest file for only specific workflows, protocols, or process types, follow the steps outlined previously, using -o custom instead of -o example.
When using this operation provide additional parameters (-w, -pr, and/or -pt) specify the exact entities for which to create a manifest.
For example:
Specifying every option is not required. It is also possible to specify more than one of each kind. For example, create a manifest file for a workflow with the following command:
Or for two protocols like so:
Or for two process types and a protocol with this command:
Step 2: Edit Manifest File
The next step is to edit the manifest file, removing unnecessary information and preserving only the custom configuration to import into the destination system.
Example 1: In this example, everything is deleted from the manifest file, except for the two new process types to export.
Example 2: In this example, the manifest file contains definitions for some new reagent types:
Step 3: Export to XML Configuration Package File
Copy and paste the following code onto the command line. Edit the variables (version, server IP address, username, password, and manifest and package file names) as required.
An edited command might look like the following example:
This step generates a data file in an XML format (newconfiguration.xml in our example) that is compliant with the Rapid Scripting API.
IMPORT: Installing a Configuration Package File
Prerequisites
A configuration package file has been exported. This example uses a file named newconfiguration.xml.
Access to the Config Slicer tool on the destination server has been granted.
There are no in progress steps for any of the protocols that are going to be sliced in, otherwise the import of the protocol fails.
Step 1: Import Configuration Package File
On the destination server, copy and paste the following code to the command line. Edit the variables (version, package file name, server IP address, username, and password) as required.
A command with the variables filled in might look like this:
About duplicate entities
If any of the configuration entities that are about to import, exist in the destination system, Config Slicer either logs a warning or attempts to update them. It depends on the mode being run (see Import/Export Mode).
If Clarity LIMS has maintained an internal record of deleted items, the previous information may also apply to deleted entities. This situation may occur if those entities have created outputs that currently still exist in the system.
When running in import mode, entities that exist and are different from the version in the package have a warning and full diff logged.
When running in importAndOverwrite mode, Config Slicer attempts to update entities that exist and are different from the version in the package.
In this scenario, back up configuration package containing copies of the updated entities before they were changed is saved to the directory where the configuration package is located. If that directory is not writable, the backup package is saved to the same directory as the log file.
If the version in the package is identical to the version on the server, no errors are logged and Config Slicer considers that entity successfully imported.
To avoid changing existing configuration (which could possibly break historical data), another option is manually renaming the old entities. Add an extension or a prefix and continue with importing the new configuration package.
Step 2: Validate the import
Use the following methods to validate whether an import has completed successfully:
Check the Import Log:
For each specific type of configuration that is being imported (e.g. container types, process types, workflows), Config Slicer will log a set of messages. The messages look similar to the following examples:
Before it begins to process a specific entity, the file logs how many entities were found. Any errors or warnings about this set of entities always appear between Found 4 $Entities and Summary of $Entities.
Every entity that is found in the configuration package always appears in the summary, in one category or another. If a scenario occurs where this isn't true, or where the initial count of entities does not match the number in the summary, something has gone wrong and a bug report should be filed.
Validate with Config Slicer:
Running Config Slicer with the validate operation checks every entity in the package to see if it exists on the destination server. It reports results in a format similar to the log format shown previously.
Run the validate operation before or after importing:
Before importing—checks if there could be any problems when importing a configuration from a package. This feature is its primary use as it makes sure that during import, "configuration exists in package but not on server" is not considered an error case.
After importing—makes sure that the results are what was expected.
Example of validate output:
Check Configuration on the Destination Server:
The ultimate test of whether configuration has imported successfully is to check the configuration on the destination server itself. Make sure it looks and behaves as expected.
Configuration can be checked either via the Configuration screen in the Clarity LIMS user interface, or via the configuration endpoints in the API.
Guidelines and Semantics for Creating Manifest Files
Top-level entities
To be included in a configuration package file, the top-level entities of the custom configuration set must be explicitly enumerated.
Some of the top-level entities are discrete 'self-contained' units, and do not include other units (for example, container types, reagent types, artifact groups, and non-artifact/non-process type UDFs).
Some top-level entities (process types, for example) automatically include other units (refer to Non-Top-Level Entities for more information).
For process types, only configured processes, vanilla Transfer processes, Pool Samples (since 7.5), and Add Multiple Reagents (since 7.6) processes are supported. All process type details are exportable/importable.
Non-Top-Level Entities
Some entities are only included as part of other entities.
For example, process templates, process type UDFs, and artifact UDFs are only included when included in a top-level process type. (The latter is a special case, given that the same artifact UDF can be used by multiple process types.)
Required Entities
Some entities may be required by other entities. In these cases, make sure that these entities are exported/imported in the correct order.
For example, because process types may require the existence of a container type, create the container type first.
Required entities are not automatically included. If they do not exist in the destination system, explicitly include them in the manifest file.
For example, suppose that a process type declares a particular container type as a default output plate. If that container type does not exist in the destination system, include that container type in the manifest file.
Import/Export Mode
Import modes affect the transactability of the tool, allowing it to make incremental changes if errors occur or provide an all-or-none option. For example, use the operation validate mode to determine if errors were encountered.
Best-Effort Mode
This is the default import mode.
This mode attempts to import as many units as possible. Any failures are logged, but the import operation is not interrupted.
For example, a failing container type import will not prevent other container types from being processed.
Similarly, if a process type fails import because it already exists, any UDFs and process templates for that process type will still be processed.
There is no need to enter an option for this mode.
Strict Mode
If this mode is used, the import operation is aborted if it encounters a failure.
For example, if there is an API version mismatch, the operation will abort and no further imports will be executed. Note that any changes already performed are not reverted.
Use the -Strict option to enable this import mode.
Validate Mode
Use the validate operation (instead of import) to enable this mode.
This mode produces a report listing showing the following items:
Entities that would be successfully imported because they do not exist on the target server.
Entities that already exist on the target server and are identical.
Entities that already exist on the target server but are different.
Validate mode can only detect a limited set of errors. For example, it can check if a particular piece of configuration already exists. If so, it checks if it is identical to the one included in the configuration package.
This information can help determine if the importAndOverwrite option is needed instead. For details, see Command-line Options and Usage.
Example of console output:
Example Mode
Use this mode to generate a manifest file if the configuration you want to export is not tied to a specific set of workflows, protocols, or process types.
Use the example operation (instead of import) to enable this mode.
Last updated