This section describes how to add and configure the three types of automations in Clarity LIMS: step automations, project automations, and derived sample automations.
To access the Automation configuration screen, the Configuration:update permission is required. Users who do not have the Configuration:update permission do not see the Automation tab.
By default, only the Administrator role has the Configuration:update permission. For more on user roles and permissions, see User Roles and Configured Role-Based Permissions.
View existing automations and their details
Edit automation details
Delete automations
On the main menu, select Configuration.
On the configuration screen, select the Automation tab.
The , , and tabs include a collapsible Tokens list.
This list is context-sensitive. Its contents differ depending on the tab being viewed. Copy and paste tokens from these lists directly into the Command Line field.
You can create three types of automations in Clarity LIMS: step automations, project automations, and derived sample automations.
Step automations—Actions that are triggered when running samples through a step. Configure them to be triggered automatically (at the start/end of the step, or when a particular screen is entered or exited), or manually (when selecting a button on the Record Details screen). The automations are enabled on the master step, but the trigger points are configured at the master step or step level. See
Project automations—Actions that users can run on submitted samples, directly from the Projects & Samples screen. For example, you might configure an automation that gives the ability to assign the samples to a workflow. See
Derived sample automations
Step automations are reusable. After you have created an automation, you can enable it on multiple master steps.
If you intend the automation to be triggered manually, the name you choose for the automation is used to name the button that initiates it from the LIMS interface.
Two step automations can have the same name as long as they are unique in some other way. For example:
On the main menu, select Configuration.
On the configuration screen, select the Automation tab.
On the Step Automation tab, select New Automation.
The Automations configuration screen includes a Template Files section that allows for the upload of a template file to an automation. Reference the file in the automation command line and use it to generate a file that is attached to the step—typically a sample sheet file that can be used to start the instrument run.
A token for the template file is automatically added to the Tokens list. When included in the command line, the token is replaced with the absolute path of the template file at run time.
Downloadable sample sheet template files are available for several Illumina instrument integrations. For details on modifying the example template for the needs of your lab, refer to the Lab Instrument Tool Kit section of the Clarity LIMS Integrations and Tool Kits documentation.
In the Template Files section, select Upload File.
In the Upload File dialog, select Choose File, and then browse to and select the appropriate template file.
Select Upload. The file is attached to the automation and listed in the Template Files section. When upload is complete, a new dynamic token is added to the Tokens list.
You can also attach template files to automations via the API, using the files endpoint. For details, refer to the Clarity LIMS API documentation.
On the Derived Sample Automation tab, select New Automation.
In the Automation Details area, complete the required fields:
Type a name for the new automation.
The following examples show how derived sample automations can be used in the lab.
In this example, when the Requeue Samples automation is run, the following occurs:
The custom script, requeue.py, is initiated. (This script is used for illustrative purposes only and is not provided with Clarity LIMS.)
This example shows the configuration of an automation named User Input X, which includes a {user input} token in the command line. For details, refer to Step Automation Tokens in the Clarity LIMS API documentation section.
On the Projects dashboard, run the automation on selected derived samples.
On the Project Automation tab, select New Automation.
In the Automation Details area, complete the required fields:
Type a name for the new automation.
The new project automation is added to the Project Automations list, and is now available to be run on submitted samples from the Projects & Samples screen.
On the Automation Configuration screen, select one of the following tabs:
Step Automation
Project Automation
When editing step automations, keep the following in mind:
Changes you make to a step automation are reflected on all future steps on which that automation is enabled.
Steps that have already been run are not affected by changes you make to a step automation.
On the Automation Configuration screen, select one of the following tabs:
Step Automation
Project Automation
Information about deleted automations is saved in the Clarity LIMS database for historical purposes. However, there is no way to restore a deleted automation for use in Clarity LIMS.
On the Automation Configuration Screen, select one of the following tabs:
Step Automation
Project Automation
Derived Sample Automation
All step, project, and derived sample automations configured in the system are listed alphabetically on their respective tab.
Select an automation to view its details in the Automation Details area on the right.
In the Automation Details area, the following information is available:
The automation name.
Channel Name: The channel used for the automation (for more information, refer to Automation Channels in the Clarity LIMS API documentation section).
Command Line: The command line that is run when the automation is triggered.
Tokens list: A list of parameters that can be used in the command-line string.
Automation Use: The master step on which the automation is enabled (applies to step automations only.
Template Files: One or more files that are referenced in the command-line string and used to generate a file at run time (see ).
channel name is unique, or
command line is unique, or
run-program-per-event values are unique (available in the API only)
Attached files and associated master steps are ignored in these comparisons.
Two project automations cannot have the same name, regardless of the uniqueness of channel name and command line.
You cannot enable multiple automations with the same name on a master step, even if the automations are configured differently.
Type a name for the new automation.
In the Channel Name field, enter the channel to be used for this automation (for more information, refer to Automation Channels in the Clarity LIMS API documentation section).
In the Command Line field, enter the command line to be run when the automation is triggered. Copy/paste tokens from the Tokens list, as required. For details, refer to Step Automation Tokens in the Clarity LIMS API documentation section.
[Optional] Enable automation on steps:
In the Automation Use section, select inside the Enable on these Master Steps field and select the master step on which to enable the automation. (Note that this configuration is bidirectional—when configuring a master step, you can select automations to associate with that master step.)
If necessary, you can:
Repeat this process to enable the automation on multiple steps.
Select the X button to remove a step from the field.
Select Save.
The new step automation is now available to be configured on the selected master steps.
In the Command Line field:
Include a script that generates the output file.
Provide the template file token as a script parameter. You can copy and paste the token directly from the Tokens list. At run time, the token is replaced with the absolute path of the file.
Select Save.
In the Step Automation list, an icon indicates that a file is attached.
If necessary, you can:
Repeat this process to attach additional files to the automation.
Select the X button to remove the file from the automation.
In the Command Line field, enter the command line to be run when the automation is triggered. Copy/paste tokens from the Tokens list, as required.
For details, refer to Derived Sample Automation Tokens in the Clarity LIMS API documentation section.
Select Save.
The new derived sample automation is added to the Derived Sample Automations list, and is now available to be run on derived samples from the Projects dashboard.
The command-line parameters are interpreted by the automation worker node, and their values are provided to the script.
The selected samples are requeued.
On the Projects dashboard, run the automation on selected derived samples.
As the script runs, the status is reported back.
Refer also to The Projects Dashboard, [#run-automations-on-derived-samples](../dashboards/projects-dashboard.md#run-automations-on-derived-samples "mention").
A custom script called custom_script.sh is initiated.
The script prompts for a value to be entered for the input_x parameter.
The command-line parameters are interpreted by the automation worker node, and their values—along with the user-supplied input_x value—are provided to the script.
The script runs.
In the Command Line field, enter the command line to be run when the automation is triggered. Copy/paste tokens from the Tokens list, as required. For details, refer to Project Automation Tokens in the Clarity LIMS API documentation section.
Select Save.
In the list of automations on the left, select the automation to edit.
Make your changes and select Save.
In the list of automations on the left, select the automation to delete.
Select Delete.
Use an automation with the copyUDFs script and to copy custom fields from a step input to a step output. This example uses the Library Normalization master step, and shows how to copy the Concentration field from the step input samples to the output samples.
On the Lab Work configuration screen, select the Automation tab, then select the Step Automation tab.
Add a new automation.
Name the automation and enter the channel name.
In the Command Line field, copy the following command, replacing the { } placeholders with your own information:
bash -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar script:copyUDFs -u {username} -p {password} -i {processURI:v3:http} -f Concentration"
Return to the Lab Work configuration screen and select the Lab Work tab. In the Master Steps list, select the master step on which you enabled the automation.
In the Automation section, the new automation is listed. Configure as follows:
Trigger Location: Record Details
Trigger Style: Automatic upon entry
NOTE: Automation triggers can be configured at the master step or the step level. If configured on the master step, the trigger settings are locked on all steps derived from that master step.
You can add it as an expanded view field or as a column header (for details, refer to ).
Save your changes.
At run time:
When the Record Details screen is entered, the automation are automatically triggered.
The copyUDFs script runs and copies the Concentration field values from the step input samples to the output samples.
In the Automation Use section, enable the automation on the desired master step (this example uses the Library Normalization master step).
Save the automation.
When Clarity LIMS is running scripts via the External Program Plugin mechanism, it is not uncommon for these scripts to rely upon a file that contains information germane to the script. A common example would be using the sample input file generator script that is part of the Lab Instrument Toolkit. This script merges runtime information within a Clarity step into a file whose format is directed by a 'template' file.
Under the old method, template files must be saved to a folder accessible to the automation worker node. Typically
If a script needs a template file, the file is specified by including its full path in the syntax that invokes the script.
A single template file might be associated with many automations. If the template file needs a minor update, the update can be made in one place. You do not need to update the individual automations.
Creating a template file is typically an iterative process: create the file, run the script, examine the output, tweak the template file, run the script, examine the output, tweak the template file, and so on.
The file must live on the server to be accessed by the script. Therefore, the developer needs access to the server filesystem—or must go through a trusted party to get the file to the server. This process inhibits rapid development and testing.
As of Clarity LIMS v5.1, template files can (optionally) be attached directly to an automation via the GUI.
The iterative process—create file, run script, examine output, tweak template file, run script, examine output, and so on— is much faster because the template file can be replaced via the GUI in a matter of seconds.
If the workflow is being validated, the validator can download and inspect the template file directly from the GUI. There is no need to gain access to the server filesystem.
Because a single template file is associated with a single automation, a template file with common contents can be duplicated and associated with many automations. If the template needs modification, it must be modified in many places.
When configuration that involves template files is migrated from one Clarity LIMS system to another using the config-slicer tool, the template files themselves are not migrated. After the config-slicer has done its work, the administrator must update the automations in the Clarity LIMS GUI, reuploading and associating the template files. If these steps are not done correctly, the syntax invoking the scripts breaks because the required files are not present on the destination system.
Because a template file is associated with a specific automation, the system allows for multiple automations associated with template files that all have the same name for the template file. While this configuration is not a problem in itself, it can cause problems. For example, if converting from the new method to the old method, you might have multiple template files with the same name but different contents. These files cannot be copied to /opt/gls/clarity/customextensions/ because they overwrite each other. Instead, each one needs to be placed in a distinct folder: /opt/gls/clarity/customextensions/folderName/. (See later why you might want to convert from new method to old.)
We recommend that you use a combination of both methods, as follows.
Use the embedded template while developing the template. During this process, having the template file easily available for editing is helpful. After the template is finalized, move it to the server and adjust the automation command line to use the server path/filename instead of the file token.
This method allows for easy, iterative testing and precise traceability for production work. This method also facilitates reliable migrations involving the config-slicer tool and coordinated movement of associated /customextensions/ files.
Config-slicer does not currently migrate automations that need template files without additional manual manipulation after the configuration migration. Regardless of method, you must manipulate the system manually to complete the migration of the template files.
/opt/gls/clarity/customextensions/folderName/templateFileNameTemplate files can be easily edited and overwritten by an administrator. This interaction might be undesirable in a validated environment. It might be safer if the templates files are harder to reach, and thus not available for inadvertent modification.
Automations (formerly referred to as EPP triggers or automation actions) allow lab scientists to invoke scripts as part of their workflow. These scripts must successfully complete for the lab scientist to proceed to the next step of the workflow.
EPP automation/support is compatible with API v2 r21 and above.
The API documentation includes the terms External Program Integration Plug-in (EPP) and EPP node.
As of BaseSpace Clarity LIMS v5.0, these terms are deprecated. The term EPP has been replaced with automation. EPP node is referred to as the Automation Worker or Automation Worker node. These components are used to trigger and run scripts, typically after lab activities are recorded in the LIMS.
Automations have various uses, including the following:
Workflow enforcement—Makes sure that samples only enter valid protocol steps.
Business logic enforcement—Validates that samples are approved by accounting before work is done on them. This automation can also make sure that selected samples are worked on together.
Automatic file generation—Automates the creation of driver files, sample sheets, or other files specific to your protocol and instrumentation.
You can enable automations on master steps in two configuration areas of Clarity LIMS:
On the Automations tab, when adding or configuring an automation. For more information, see .
On the Lab Work tab, on the master step configuration form. For more information, see .
After it is enabled on a master step, the automation becomes available for use on all steps derived from that master step.
You can configure the automation trigger on the master step, or on the steps derived from that master step.
While executing a script, if more than one script would be triggered for a single user action, they are reported in sequence. This reporting continues until all scripts complete, or one of them fails.
An example scenario would be a step that is configured to execute the following:
One script upon exit of the Placement screen.
You can request to cancel a script that is not responsive. While canceling abandons the monitoring of script execution, it does not stop the execution of the script.
After canceling a script, follow up with the Clarity LIMS administrator to determine if the AI node/automation worker must be restarted.
The scientific programmers in your facility can provide you with a message upon successful execution of a script. There are two possible non-fatal messages: OK and WARNING. These messages can be set using the step program status REST API endpoint.
Message boxes display the script name, followed by a message that is set by the script using the step program status REST API endpoint. Line breaks are permitted in the custom message. The following is an example of a success message:
After you select OK, you are permitted to proceed in the workflow.
Notification—Notifies external systems of lab progress. For example, you can notify Accounting of completed projects so that they can then bill for services rendered.
A second script upon entry of the Record Details screen.
In this scenario, when the lab scientist advances their protocol step from the Placement screen to the Record Details screen, the scripts are executed in sequence.
The parameter string/automation name configured on the master step is displayed in a progress message. You can use this feature by giving your parameter strings/automations meaningful names that provide you with context about what the script is doing. The following is an example of a progress message.
You cannot proceed until the script completes successfully.
For example, when beginning a step, if the script does not allow you to work on the samples together in Ice Bucket, the samples will be returned to Ice Bucket after acknowledging the error message. In this case, the step is prevented from being tracked. The following is an example of a failure message:
If you attempt to advance a step from the Pooling screen, but an error is detected, the error state prevents you from continuing. The following is an example of this type of message:
After you select OK, you are prevented from proceeding in the workflow. Instead, you must return to the Pooling screen and address the problem before proceeding.
