Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Lab Logic Toolkit (LLTK) provides the evaluateDynamicExpression script, which allows for the evaluation of simple dynamic expressions — Excel-like formulas that can be used to perform calculations. The expressions may include full Java or Groovy syntax.
Each Clarity LIMS step comprises a series of mappings that relate a single input to a single output. The script applies and evaluates the expression for each input-output-mapping in the step.
For example, the following expression sets the next step for each output sample, depending on whether the input sample has passed or failed QC:
Often, the formulas are used to set the values of one or more custom fields. However, LLTK can do a great deal more than simply manipulate field values.
When working with the Lab Logic Toolkit evaluateDynamicExpression script, custom field values can originate from the following data collection entities (these will differ, depending on the step from which the script is being run):
input (analyte/derived sample)
output (analyte/derived sample or per-input result file, measurement, or file placeholder)¹
step
container
submittedSample / submittedSamples²
¹ Shared and per-reagent-label result files / measurements (e.g., those produced by a demultiplexing step) are not supported.
² Use 'submittedSamples' with pooled derived samples, and 'submittedSample' for single-source derived samples.
Use the fail() method to halt the script and display an error message. If the script is used on a step transition, this method also prevents the step from proceeding to the next screen.
Example expression:
Note that this example also uses the hasValue function (see Handling Non-Existent UDFs/Custom Fields).
The following table summarizes the non-UDF/custom field properties available for each entity type. Column headers are the types available. The bold row entries are the properties available.
LIMSID is supported via Groovy node access. See limsid section.
The name read-only property makes the name of the entity available to the scripting engine.
Example basic expression:
The workflowName read-only property makes the workflow name of input samples available to the scripting engine.
This property contains a string representing the current IN_PROGRESS workflow names. The string is formatted as a comma-delimited list, with a space inserted before each new item.
Example basic expressions:
The replicateCount read-only property is available for input samples.
This property contains an integer value representing the number of output replicates (samples or result files / measurements or files / file placeholders) generated from the input sample in the current step.
Example basic expressions:
The number of input analytes/derived samples that contributed to this analyte / derived sample.
For pooled analytes / derived samples, this is > 1.
For non-pooled analytes this is 1.
Replicated samples that are then pooled are still counted separately in pooledInputsCount.
Example basic expression:
When working with the pooledInputsCount property, the script looks at the step that created the derived sample or result file / measurement, and doesnot look upstream for a pooling step.
Example scenario:
Step 1: Pooling step that creates pooled samples.
Step 2: Nonpooling step that takes in the pooled samples and creates new derived samples (that are themselves still pools).
Step 3: Nonpooling step with expression that calls input.pooledInputsCount.
The expression in step 3 returns '1' because it onlys look at the previous step (step 2)—the step that created the current derived samples.
You can access the limsid field with some knowledge of Groovy and the LLTK. Use the following examples with any supported entity:
Example basic expressions:
The well read-only property makes the well location of input, output, or submittedSample available to dynamic expressions. Result file / measurement outputs are supported if they have placement information (eg, on QC steps).
Example basic expression:
The following table defines the parameters used by the evaluateDynamicExpression script.
Use the following special syntax to join multiple evaluations within an expression:
AND = &&
OR = ||
NOT = !
Boolean expressions should evaluate to true or false.
For debugging errors, run through the following steps to determine the cause.
Check custom field format (eg, step.::UDF Name::).
Make sure that the custom fields have been created for the correct entity type.
Check the mapping of the field type (see section) — if doing Math they need to be numeric.
Check that the step has a log file placeholder configured.
The 'unexpanded placeholders in string' error often refers to the last -log parameter — where the step has not been configured to produce one.
If no log file is created, find additional error logging from the command line of the instance. This often shows the location of a syntax error. Find the log file at /opt/gls/clarity/automation_worker/node/logs/automatedinformatics.log.
Set -t parameter to false if an automatic trigger is used.
BaseSpace Clarity LIMS Lab Logic Toolkit (LLTK) provides the evaluateDynamicExpression script, which allows for the evaluation of simple dynamic expressions.
This article provides examples that you can modify to suit the needs of your lab.
For details on script parameters and usage, and additional examples, see the article.
See the article.
See the article.
This example assumes that the step UDF / custom field Total samples has a default numeric value of 0:
Without a default value, the expression must set the initial value before continuing to sum:
This example demonstrates the use of step UDFs / custom fields to store temporary values in order to enable calculations between iteration.
This example checks that the container name has been changed from the default value (container LIMSID)
Check the flow cell/reagent cartridge format (Illumina’s flow cell check) as follows:
The following expression results in the format: Wed Apr 03 21:28:38 EDT 2015
The following expression results in the format: 2015-04-03
This example assumes that the analyte / derived sample UDF Pool Volume (uL) has a default numeric value of 0.
This can also be done without setting a default value for the UDF:
The example below assumes the following:
The placement is alphanumeric (e.g., A:1)
The offset for the alphabetical row is 0, and the offset for the numeric column is 1
The placement order desired is horizontal: A:1 maps to 1, A:2 maps to 2, etc.
The placement of interest is the output placement
Placement is configured for a specific, known container type, and the dimensions of this container type are also known. This example uses a 96 well container configuration (12 columns are available and this value is used directly in the expression below). This value can be replaced for other container types.
Note the following:
The call to split() here will return a list of row:column, so we use [0] and [1] to access these values, respectively.
:: : ::.trim() is used to obtain the result ':' - because :: is used as a reserved character, and thus ::::: directly results in '': and errors.
In some cases, special characters are not permitted in a field in the LIMS. To prevent users from entering special characters, the best practice is to include validation to check that the field contains only letters and/or numbers. Wherever possible, we recommend that you use this 'positive checking' method as it is much less error-prone. However, if this preferred method is not possible, you can use the Lab Logic Toolkit to check for the presence of special characters, and display an error message if one is found.
The following example use the Groovy matches command to check step output container names for the characters ? ( ) [ ] / \ and quotes or spaces.
The matches command
The matches command takes a regular expression (regex) as its input. Call this command on the field you want to check.
In our example, the matches command is:
If we isolate the regex, we have:
Here, we're checking for any text that contains any one of the special characters listed. It's important to note that the way regex is supported in Java (and therefore Groovy) leads to some quirks, noted in the following table.
Escaping characters Normally, in regex we would need to escape the following characters using a single backslash:
However, because of how matches works, not all of these characters need to be escaped - for example the '?' character. Characters that do still need to be escaped, need to be escaped with an additional backslash (e.g., \\).
Checking for quotes Because the command is run via Automated Informatics / Automation Worker, we also need to use a different approach for checking for quotes.
We cannot use them as we typically would when writing a regex expression (e.g., "), or escape them as we would in a call to matches (e.g., \"). Instead, we need to provide them as their unicode number, shown in our example as \u0022 and \u00.
The following tables summarize the mapping of UDF/custom field types.
The LIMS and API treat the toggle switch field type as boolean when its value is set to true or false. However, it is possible to configure a toggle switch to not have to set a boolean true / false value. In this case, the value of the field is None Set and the field is treated as a 'nonexistent' field. To make sure that a toggle switch value has been set, you can use the hasValue function. For details, see .
Each custom field is specified by using one of the data collection entity names (see ), followed by a dot separator, followed by the field name enclosed in :: characters.
For example:
The expression is evaluated and the target field value set for each input/output pair in the step.
Where there are replicates or pooling this may mean that a particular input or output item is evaluated multiple times.
Lab Logic Toolkit can write to read-only fields.
Lab Logic Toolkit can write to all hidden fields.
The evaluateDynamicExpression script evaluates the expression and sets the target field value for each input/output pair in the step.
Where there are replicates or pooling this may mean that a particular input or output item will be evaluated multiple times.
The output data collection is treated specially. The script performs the following actions:
Searches the per-input output sample and the per-input output result file/measurement (if they exist) for any output field names found in the expression.
Returns any values found.
In the event of a name collision (ie, the same field name exists on both sample output and result file / measurement output) the sample value is used.
Example 1
In this example expression, the sample output DNA Concentration field is assigned the value resulting from the following calculation:
Multiplying the value of the step Ideal Volume by the value of its Ideal Concentration.
Dividing that result by the Volume defined on the sample input.
Note the following:
step.::Ideal Volume:: is the value of a UDF/custom field. The period between step and Ideal Volume signifies that Ideal Volume is associated with the step.
The part of the expression to the left of the = sign is always the destination or result of the expression, that is:
The part of the expression to the right of the = sign is always the source of the value to be stored in the destination, that is:
In this case, multiplying the value of the protocol step Ideal Volume by the value of its Ideal Concentration, and then dividing the result by the Volume defined on the sample input.
Example 2
Copy a step or analyte / derived sample field value to a field configured on the step output - for use in downstream steps:
This expression will be evaluated one time for each output.
Example 3
Copy a result file or measurement field value to the output analyte / derived field for use in downstream step:
Example 4
Automatically set a text-string field value:
In the API, fields that do not exist and fields that do not have a value set are considered the same and are handled in the same way.
Attempting to access a nonexistent UDF / custom field will usually result in an error. Sometimes this may not be the desired behavior.
Before attempting to access a field, you can use the hasValue function to check if it exists.
The hasValue function:
May be used on any item.
Takes one parameter - the name of the field.
Returns true if the field exists and has a non-null value.
Example Expressions:
The following expression checks that the Concentration field has a non-null value, and then checks that this value is greater than 1:
The following expression checks that a toggle switch field has a non-null value - ie, that it is set to true or false, and then checks that the value is set to true:
The nextStep property is similar to the input, output, submittedSample, container, and step entities. However, nextStep can only be used on the left side of a dynamic expression; it cannot be used in the calculation on the right side of the expression.
The basic syntax is as follows:
Depending on how the step is configured, a nextStep expression may set the next action for the step inputs or the step outputs. For example, in steps that do not produce output samples — such as QC steps, it is the step inputs that proceed (provided they pass QC) to the next step in the workflow.
When creating an expression that calls the nextStep property, there is no need to differentiate between step inputs and step outputs. The LIMS is able to determine this from how the API represents the entities that are available to set next actions on.
Note also that this property is evaluated for each input/output, and can set a different nextStep value for each.
For samples to advance to the next step via an LLTK expression, the next step must be specified in the Configuration area of the LIMS.
Configure next steps on the protocol configuration form, in the Next Steps table.
The possible next step values listed in the following table are intended for non-QC protocols. There is no nextStep entity for a QC protocol. However, samples in a QC step in a QC protocol can move to REMOVE or ESCALATE via LLTK.
<next step value> can be one of the following:
Any Groovy expression that includes the top-level entity UDFs/custom fields on input, output, submittedSample, and evaluates to one of the options described.
A valid next step name, for example, ::Sample Pooling:: (remember that text / string values must be inside double colon delimiters).
One of the following 'special values' described in the following table.
The following conditions must be met:
If <next step value> is not a valid next step, or is not one of the special values described in the table above, an error is raised and the script will fail by returning a nonzero value.
The <next step value> must be in 'choose next steps' or 'record details' transition when the script is executed. If not, the script will fail and will return an error message.
Example expressions:
When working with control samples, single-use control samples cannot have their next steps set to anything other than REMOVE.
Supplying the optional -excludeControls parameter with a value of true ensures the following:
The script calculation is only evaluated for noncontrol samples.
If next steps are set, the value is set to REMOVE for control samples.
Note also that the Java contains() method provides a simple way of identifying controls to be removed:
In QC steps, some items have a QC flag associated with them. This flag can be read and set from script expressions. The value may be set to true or false.
Example expressions:
Workflows are not assigned to output samples until the step that creates those samples completes (this will always be the current step). For this reason, the workflow name should always be retrieved from the input entity.
Because the replicateCount property is only available for input samples, its value for output samples will always be zero.
This example includes , using both implicit variables (it) and explicit variable names (for example, currentSample).
See the article.
Property
Input Sample
Output Sample
Per Input Result File/Measurement
Submitted Sample
Container
Step
name
yes
yes
yes
yes
yes
yes
container
yes
yes
yes
yes
no
no
workflowName
yes
yes
no
no
no
no
replicateCount
yes
no
no
no
no
no
pooledInputsCount
yes
yes
yes
no
no
no
well
yes
yes
yes
yes
no
no
Parameter
Description
-u {username}
(Required) LIMS login username
-p {password}
(Required) LIMS login password
-i {URI}
(Required) LIMS step URI Provides context to resolve all token values.
-t {true/false}
(Optional) Test mode The script may be executed automatically on step transition, or manually — by selecting the button on the Record Details screen.
For automatic script execution—set test mode to false.
For manual script execution—set test mode to true.
Defaults to false if not specified.
-h {true/false}
(Optional) Halt on any error. By default, if the expression fails to evaluate, or evaluates to a non-numeric/non-sensical value, the script logs the problem and continue. A warning message advises the user to review the generated log file. Supplying the optional -h parameter with a value of true will cause the script to fail, preventing a step transition from proceeding, if any nonfatal errors arise.
-exp {expression}
(Required) The expression to be evaluated. The expression must be enclosed in single quotes, eg, -exp 'expression'. See Using Special Characters - Rules and Constraints section.
-log {logFileLIMSID} -logFileName {logFileLIMSID}
(Required) The LIMS ID of the log file placeholder
-excludeControls {true/false} (Cannot be used withonlyControls; these two parameters are mutually exclusive)
(Optional) Control sample handling Set this parameter to true to make sure that:
The nextStep (if used) is set to REMOVE for control samples.
Controls are excluded from any calculations in the expression.
-onlyControls {true/false} -onlyControlSamples {true/false} (Cannot be used withexcludeControls; these two parameters are mutually exclusive)
(Optional) Control sample handling Set this parameter to true to make sure that calculations are performed on control samples only.
-s {true/false} -stopwatch {true/false}
(Optional) Tool logging Set this parameter to true to record the time needed to run the tool as an entry in the log file.
Special Character
Symbol
Rules and Constraints
Single quote
'
As the ‘exp’ argument supplied to the script must be enclosed in single quotes, the single quote character cannot be used anywhere in the expression—including as part of any custom field that contains the single quote in its name.
Double quote
"
The entire command line must be enclosed in double quotes. Therefore, the double quote character cannot be used anywhere in the expression — including as part of any custom field that contains the double quote in its name.
Double colon
::
This two character string is used to delimit strings and custom field names. This character may not appear inside a custom field or string.
Example
Description
A; B
Run A and then B, regardless of success of A.
A && B
Run B if A succeeded.
A || B
Run B if A failed.
if (!A) {B}
If A condition is not met, run B.
Entry | Standard regex representation | Character being checked for |
\u0022 and \u0027 | ' and " | Single and double quotes |
? | \? | Question mark |
( and ) | \( and \) | Brackets |
[ and ] | \[ and \] | Square brackets |
/ | \/ | Forward slash |
\ | \\ | Backslash |
(space) | \s | Spaces |
Field Type | Groovy Data Type |
Text | String |
Multiline Text | String |
Text Dropdown | String |
Hyperlink | String |
Hyperlink Dropdown | String |
Toggle Switch | Boolean |
Date | Date |
Numeric | double, float, log, int, BigDecimal, BigInteger |
Numeric Dropdown | double, float, log, int, BigDecimal, BigInteger |
UDF/Custom Field Specification | Description |
input.::Concentration (ng/mol):: | The value of the field Concentration (ng/mol) from the input sample |
submittedSample.::Batch ID:: | The value of the field Batch ID from the submitted sample |
step.::Priority in Process:: | The value of the field Priority In Process on the current step |
Special Value | Description |
::ADVANCE:: | Used to move the samples to the logical next step in the protocol (provided only one next step is permitted) |
::REPEAT:: | Used to select Repeat this step. |
::REMOVE:: | Used to remove sample from workflow. |
::ESCALATE:: | Used to select Request Manager Review. Reviewer name is left unset. |
::COMPLETE:: | Used to select Mark protocol as Complete. |
::COMPLETE_REPEAT:: | Used to select Complete and Repeat this step. |
::COMPLETE_REPEAT:<step name>:: | Used to select Complete and Repeat option, selecting the named step as the next step to continue with. |
::REWORK:<step name>:: | Used to select Rework to a previous step, selecting the named step as the step to rework the samples from. |
For a pooled derived sample consisting of several submitted samples, Lab Logic Toolkit exposes a submittedSamples property that is a collection of the submitted samples represented in the well.
submittedSamples returns a list of submitted samples.
If it is not a pooled sample, it will return a list of 1.
submittedSample returns an error if sample is pooled.
Use this collection to read and/or update all related submitted samples.
For example:
To set the sample level UDF/custom field for each of the related submitted samples:
To sum the Volume UDF/custom field on all the related submitted samples into a single value written to the input analyte / derived sample:
For information on Groovy syntax and closures, see Groovy Closures.
The container entity makes the input, output, or submittedSample container available to the scripting engine.
The container can be queried for its name, for example:
Container name, UDFs / fields, and entity information can be read or written, for example:
Example basic expressions:
The word 'container' may be optionally surrounded by "::"
Lab scientists need to record both a start and stop date/time for an incubation. As a second evaluation of how many hours the incubation lasted, the scientists want BaseSpace Clarity LIMS to calculate the elapsed time.
The Lab Logic Toolkit evaluateDynamicExpression script can compare date/time entries and calculate elapsed hours. As Date fields in the LIMS do not display the required time component for this calculation, Text fields are needed.
The bash command uses a 'Date' function to convert the Text date/time entries into dates, and also uses a 'getTime' method to convert those dates into milliseconds since 1 January 1970 00:00:00 UTC. The difference between the two converted entries is then changed from milliseconds to hours.
Create a step level Text field for the Start Date/Time.
In this example, the field is named Incubation Start yyyy-mm-dd hh24:mi. The format is included in the name to assist the scientist entering the date/time.
Create a step level Text field for the Stop Date/Time.
In this example, the field is named Incubation Stop yyyy-mm-dd hh24:mi.
Create a step level Numeric field for the Incubation Hours. Set your preferred decimal place limit.
Create an automation and enter the following command line (if you have used different field names, edit the example command line to match):
Set the automation to be triggered by a button on the step. This allows the scientist to enter/edit the Start and Stop Date/Time before the elapsed hours are calculated.
Adjustment for the Start/Stop events taking place over the 'daylight savings change events' is not included in this example.
If you want to use a button to populate a Text field with the current date/time, then use:
You can use all methods included in the Java and Groovy base packages.
Commonly used Java classes are:
String: http://docs.oracle.com/javase/8/docs/api/java/lang/String.html
Math: https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html
For information on Groovy closures, see .
Use the hasValue function to check for nonexistent fields and fields that do not have a value set - including toggle switches in LIMS v5 and later.
See .
Use the hasValue function together with the fail() method to check for step UDF / custom field values when the Record Details screen is exited, and prevent the user from advancing to the next screen if they have not entered values for these fields.
See .
Use the following expression:
Use Groovy syntax to format the following string:
Replace step.::counter:: with the field you want to apply the formatting to.
Replace %.0f with the number of decimal places needed. For example, for three decimal places, use %.3f.
Example:
Use the following expression:
Custom fields are set as a bulk group in the API. The API checks all required fields.
When LLTK updates a field, all required fields must be set or an error will occur.
Workaround:
Set the required fields with a value such "Please fill in this value".
Use another LLTK expression triggered on step completion to check whether this value has been updated.
If the field value has been updated, allow the step to proceed.
If the field value has not been updated, log an error and halt progress of the step.