Automations (formerly referred to as EPP triggers or automation actions) allow lab scientists to invoke scripts as part of their workflow. These scripts must successfully complete for the lab scientist to proceed to the next step of the workflow.
EPP automation/support is compatible with API v2 r21 and above.
The API documentation includes the terms External Program Integration Plug-in (EPP) and EPP node.
As of BaseSpace Clarity LIMS v5.0, these terms are deprecated. The term EPP has been replaced with automation. EPP node is referred to as the Automation Worker or Automation Worker node. These components are used to trigger and run scripts, typically after lab activities are recorded in the LIMS.
Automations have various uses, including the following:
Workflow enforcement—Makes sure that samples only enter valid protocol steps.
Business logic enforcement—Validates that samples are approved by accounting before work is done on them. This automation can also make sure that selected samples are worked on together.
Automatic file generation—Automates the creation of driver files, sample sheets, or other files specific to your protocol and instrumentation.
Notification—Notifies external systems of lab progress. For example, you can notify Accounting of completed projects so that they can then bill for services rendered.
You can enable automations on master steps in two configuration areas of Clarity LIMS:
On the Automations tab, when adding or configuring an automation. For more information, see #add-and-configure-automations.
On the Lab Work tab, on the master step configuration form. For more information, see #add-and-configure-master-steps-and-steps.
After it is enabled on a master step, the automation becomes available for use on all steps derived from that master step.
You can configure the automation trigger on the master step, or on the steps derived from that master step.
This section describes how to add and configure the three types of automations in Clarity LIMS: step automations, project automations, and derived sample automations.
To access the Automation configuration screen, the Configuration:update permission is required. Users who do not have the Configuration:update permission do not see the Automation tab.
By default, only the Administrator role has the Configuration:update permission. For more on user roles and permissions, see User Roles and Configured Role-Based Permissions.
You can create three types of automations in Clarity LIMS: step automations, project automations, and derived sample automations.
Step automations—Actions that are triggered when running samples through a step. Configure them to be triggered automatically (at the start/end of the step, or when a particular screen is entered or exited), or manually (when selecting a button on the Record Details screen). The automations are enabled on the master step, but the trigger points are configured at the master step or step level. See #add-a-step-automation
Project automations—Actions that users can run on submitted samples, directly from the Projects & Samples screen. For example, you might configure an automation that gives the ability to assign the samples to a workflow. See #add-a-project-automation
Derived sample automations—Actions that users can run on derived samples, directly from the Projects dashboard. For example, you could configure an automation that gives the ability to queue selected samples for a new workflow. In this case, the automation would trigger a custom script created for this purpose. See #add-a-derived-sample-automation
Step automations are reusable. After you have created an automation, you can enable it on multiple master steps.
If you intend the automation to be triggered manually, the name you choose for the automation is used to name the button that initiates it from the LIMS interface.
Two step automations can have the same name as long as they are unique in some other way. For example:
channel name is unique, or
command line is unique, or
run-program-per-event values are unique (available in the API only)
Attached files and associated master steps are ignored in these comparisons.
Two project automations cannot have the same name, regardless of the uniqueness of channel name and command line.
You cannot enable multiple automations with the same name on a master step, even if the automations are configured differently.
On the main menu, select Configuration.
On the configuration screen, select the Automation tab.
On the Step Automation tab, select New Automation.
In the Automation Details area, complete the required fields:
Type a name for the new automation.
In the Channel Name field, enter the channel to be used for this automation (for more information, refer to Automation Channels in the Clarity LIMS API documentation section).
In the Command Line field, enter the command line to be run when the automation is triggered. Copy/paste tokens from the Tokens list, as required. For details, refer to Step Automation Tokens in the Clarity LIMS API documentation section.
[Optional] Enable automation on steps:
In the Automation Use section, select inside the Enable on these Master Steps field and select the master step on which to enable the automation. (Note that this configuration is bidirectional—when configuring a master step, you can select automations to associate with that master step.)
If necessary, you can:
Repeat this process to enable the automation on multiple steps.
Select the X button to remove a step from the field.
Select Save.
The new step automation is now available to be configured on the selected master steps.
The Automations configuration screen includes a Template Files section that allows for the upload of a template file to an automation. Reference the file in the automation command line and use it to generate a file that is attached to the step—typically a sample sheet file that can be used to start the instrument run.
A token for the template file is automatically added to the Tokens list. When included in the command line, the token is replaced with the absolute path of the template file at run time.
Downloadable sample sheet template files are available for several Illumina instrument integrations. For details on modifying the example template for the needs of your lab, refer to the Lab Instrument Tool Kit section of the Clarity LIMS Integrations and Tool Kits documentation.
In the Template Files section, select Upload File.
In the Upload File dialog, select Choose File, and then browse to and select the appropriate template file.
Select Upload. The file is attached to the automation and listed in the Template Files section. When upload is complete, a new dynamic token is added to the Tokens list.
In the Command Line field:
Include a script that generates the output file.
Provide the template file token as a script parameter. You can copy and paste the token directly from the Tokens list. At run time, the token is replaced with the absolute path of the file.
Select Save.
In the Step Automation list, an icon indicates that a file is attached.
If necessary, you can:
Repeat this process to attach additional files to the automation.
Select the X button to remove the file from the automation.
You can also attach template files to automations via the API, using the files endpoint. For details, refer to the Clarity LIMS API documentation.
On the Derived Sample Automation tab, select New Automation.
In the Automation Details area, complete the required fields:
Type a name for the new automation.
In the Channel Name field, enter the channel to be used for this automation (for more information, refer to Automation Channels in the Clarity LIMS API documentation section).
In the Command Line field, enter the command line to be run when the automation is triggered. Copy/paste tokens from the Tokens list, as required.
For details, refer to Derived Sample Automation Tokens in the Clarity LIMS API documentation section.
Select Save.
The new derived sample automation is added to the Derived Sample Automations list, and is now available to be run on derived samples from the Projects dashboard.
The following examples show how derived sample automations can be used in the lab.
On the Project Automation tab, select New Automation.
In the Automation Details area, complete the required fields:
Type a name for the new automation.
In the Channel Name field, enter the channel to be used for this automation (for more information, refer to Automation Channels in the Clarity LIMS API documentation section).
In the Command Line field, enter the command line to be run when the automation is triggered. Copy/paste tokens from the Tokens list, as required. For details, refer to Project Automation Tokens in the Clarity LIMS API documentation section.
Select Save.
The new project automation is added to the Project Automations list, and is now available to be run on submitted samples from the Projects & Samples screen.
On the Automation Configuration screen, select one of the following tabs:
Step Automation
Project Automation
Derived Sample Automation
In the list of automations on the left, select the automation to edit.
Make your changes and select Save.
When editing step automations, keep the following in mind:
Changes you make to a step automation are reflected on all future steps on which that automation is enabled.
Steps that have already been run are not affected by changes you make to a step automation.
On the Automation Configuration screen, select one of the following tabs:
Step Automation
Project Automation
Derived Sample Automation
In the list of automations on the left, select the automation to delete.
Select Delete.
Information about deleted automations is saved in the Clarity LIMS database for historical purposes. However, there is no way to restore a deleted automation for use in Clarity LIMS.
When Clarity LIMS is running scripts via the External Program Plugin mechanism, it is not uncommon for these scripts to rely upon a file that contains information germane to the script. A common example would be using the sample input file generator script that is part of the Lab Instrument Toolkit. This script merges runtime information within a Clarity step into a file whose format is directed by a 'template' file.
Under the old method, template files must be saved to a folder accessible to the automation worker node. Typically
If a script needs a template file, the file is specified by including its full path in the syntax that invokes the script.
As of Clarity LIMS v5.1, template files can (optionally) be attached directly to an automation via the GUI.
We recommend that you use a combination of both methods, as follows.
Use the embedded template while developing the template. During this process, having the template file easily available for editing is helpful. After the template is finalized, move it to the server and adjust the automation command line to use the server path/filename instead of the file token.
This method allows for easy, iterative testing and precise traceability for production work. This method also facilitates reliable migrations involving the config-slicer tool and coordinated movement of associated /customextensions/ files.
Config-slicer does not currently migrate automations that need template files without additional manual manipulation after the configuration migration. Regardless of method, you must manipulate the system manually to complete the migration of the template files.
Use an automation with the copyUDFs script and to copy custom fields from a step input to a step output. This example uses the Library Normalization master step, and shows how to copy the Concentration field from the step input samples to the output samples.
On the Lab Work configuration screen, select the Automation tab, then select the Step Automation tab.
Add a new automation.
Name the automation and enter the channel name.
In the Command Line field, copy the following command, replacing the { } placeholders with your own information:
bash -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar script:copyUDFs -u {username} -p {password} -i {processURI:v3:http} -f Concentration"
In the Automation Use section, enable the automation on the desired master step (this example uses the Library Normalization master step).
Save the automation.
Return to the Lab Work configuration screen and select the Lab Work tab. In the Master Steps list, select the master step on which you enabled the automation.
In the Automation section, the new automation is listed. Configure as follows:
Trigger Location: Record Details
Trigger Style: Automatic upon entry
NOTE: Automation triggers can be configured at the master step or the step level. If configured on the master step, the trigger settings are locked on all steps derived from that master step.
You can add it as an expanded view field or as a column header (for details, refer to #configure-record-details-milestone).
Save your changes.
At run time:
When the Record Details screen is entered, the automation are automatically triggered.
The copyUDFs script runs and copies the Concentration field values from the step input samples to the output samples.