Workflows, chemistry, hardware, and software are continually changing in the lab. As a result, you can determine which samples were processed after a specific change happened.

Using the processes (list) resource you can construct a query that filters the list using both process type and date modified.

Prerequisites

Before you follow the example, make sure you have the following items:

• Samples that have been added to the system.

• Multiple processes of the Cookbook Example type that have been run on different dates.

• A compatible version of API (v2 r21 or later).

Code Example

In Clarity LIMS, when you search for a specific step type, the search results list shows all steps of that type that have been run, along with detailed information about each one. This information includes the protocol that includes the step, the number of samples in the step, the step LIMS ID, and the date the step was run.

The following screenshot shows the search results for the step type Denature and Anneal RNA (TruSight Tumor 170 v1.0).

The list shows the date run for each step, but not the last modified date. This is because a step can be modified after it was run, without changing the date on which it was run.

To find the steps that meet the two criteria (step type and date modified), you must to do the following steps:

1. Request a list of all steps (processes), filtered on process type and date modified.

2. Once you have the list of processes, you can use a script to print the LIMS ID for each process.

Step 1. List processes of a specific type that were modified after a specified date

To request a list of all processes of a specific type that were modified after a specified date, use a GET method that uses both the ?type and ?last-modified filter on the processes resource:

The GET call returns a list of the first 500 processes that match the filter specified. If more than 500 processes match the filter, only the first 500 are available from the first page.

In the XML returned, each process is an element in the list. Each element contains the URI for the individual process resource, which includes the LIMS ID for the process.

The URI for the list of all processes is http://yourIPaddress/api/processes. In the example code, the list was filtered by appending the following:

This filters the list to show only processes that are of the Cookbook Example type and were modified after the specified date.

The date must be specified in ISO 8601, including the time. In the example, this is accomplished using an instance of a Calendar object and a SimpleDateFormat object, and encoding the date using UTF-8. The date specified is one week prior to the time the code is executed.

All of the REST list resources are paged. Only the first 500 items are returned when you query for a list of items, such as http://youripaddress/api/v2/artifacts.

If you cannot filter the list, you must iterate through the pages of a list resource to find the items that you are looking for. The URI for the next page of resources is always the last element on the page of a list resource.

After requesting an individual process XML resource, you have access to a large collection of data that lets you modify or view each process. Within the process XML, you can also access the artifacts that were inputs or outputs of the process.

After running the script on the command line, output is be generated showing the LIMS ID for each process in the list.

When working with multiplexing, you can do the following:

• Filter Processes by Date and Type

• Find Terminal Processes/Steps

• Run a Process/Step

Information about a step is stored in the process resource. In general, automation scripts access information about a step using the processURI, which links to the individual process resource. The input-output-map in the XML returned by the individual process resource gives the script access to the artifacts that were inputs and outputs to the process.

Processing a sample in the lab can be complex and is not always linear. This may be because more than one step (referred to as process in the API and in the Operations Interface in Clarity LIMS v4.x and earlier) is run on the same sample, or because a sample has to be modified or restarted because of quality problems.

The following illustration provides a conceptual representation of a Clarity LIMS workflow and its sample/process hierarchy. In this illustration, the terminal processes are circled.

The following illustration provides a conceptual representation of a LIMS workflow and its sample / process hierarchy. In this illustration, the terminal processes are circled.

This example finds all terminal artifact (sample)-process pairs. The main steps are as follows:

1. All the processes run on a sample are listed with a process (list) GET method using the ?inputartifactlimsid filter.

2. All the process outputs for an input sample are found with a process (single) GET.

3. Iteration through the input-output maps finds all outputs for the input of interest.

Prerequisites

Before you follow the example, make sure you have the following items:

• A sample to the system.

• Several steps that have been run, with several steps run on a single output at least one time.

• A compatible version of API (v2 r21 or later).

Code Example

To walk down the hierarchy from a particular sample, you must do the following steps:

1. List all the processes that used the sample as an input.

2. For each process on that list, find all the output artifacts that used that particular input. These output artifacts represent the next level down the hierarchy.

3. To find the artifacts for the next level down, repeat steps 1 and 2, starting with each output artifact from the previous round.

This example starts from the original submitted sample.

Step 1. Retrieve the Sample Resource and Find its Analyte Artifact (Derived Sample) URI

The first step is to retrieve the sample resource via a GET call and find its analyte artifact (derived sample) URI. The analyte artifact of the sample is the input to the first process in the sample hierarchy.

The following GET method provides the full XML structure for the sample including the analyte artifact URI:

• The sample.artifact.@limsid contains the original analyte LIMS ID of the sample. For each level of the hierarchy, the artifacts are stored in a Groovy Map called artifactMap. The artifactMap uses the process that generated the artifact as the value, and the artifact LIMS ID as the key. At the top sample level, the list is only comprised of the analyte of the original sample. In the map, the process is set to null for this sample analyte.

Step 2. Find All of the Processes Run on the Artifact

To find all the processes run on the artifacts, use a GET method on the process (list) resource with the ? inputartifactlimsid filter.

In the last line of the example code, the processURI string sets up the first part of the URI. The artifact LIMSID is added (concatenated) for each GET call in the following while loop:

In the last line of the example code provided above, the processURI string sets up the first part of the URI.

The artifact LIMSID will be added (concatenated) for each GET call in the while loop below.

The while loop evaluates one level of the hierarchy for every iteration. Each artifact at that level is evaluated. If that artifact was not used as an input to a process, an artifact/process key value pair is stored in the lastProcMap. All the Groovy maps in the previous code use this artifact/process pair structure.

The loop continues until there are no artifacts that had outputs generated. For each artifact evaluated, the processes that used the artifact as an input are found and collected in the processes variable. Because a process can be run without producing outputs, a GET call is done for each of the processes to determine if the artifact generated any outputs.

Any outputs found will form the next level of the hierarchy. The outputs are temporarily collected in the outputArtifactMap. If no processes were found for that artifact, then it is an end leaf node of a hierarchy branch. Those artifact/process pairs are collected in the lastProcMap .

You can iterate through each pair of artifact and process LIMS IDs in outputArtifactMap and print the results to standard output.

Expected Output and Results

Running the script in a console produces the following output:

Steps can have user-defined fields (UDFs)/custom fields that can be used to describe properties of the steps.

For example, while a sample UDF/custom field might describe the gender or species of the sample, a process UDF/custom field might describe the room temperature recorded during the step or the reagent lot identifier. Sometimes, some of the information about a step is not be known until it has completed on the instrument, but after it was run in Clarity LIMS.

In this example, we will record the Actual Equipment Start Time as a process UDF/custom field after the step has been run in Clarity LIMS. The ISO 8601 convention is used for recording the date and time. For more information, see Filter Processes by Date and Type.

NOTE: In the API, information about a step is stored in the process resource.

As of Clarity LIMS v5, the term user-defined field (UDF) has been replaced with custom field in the user interface. However, the API resource is still called UDF.

• Master step fields—Configured on master steps. Master step fields only apply to the following:

  • The master step on which the fields are configured.

  • The steps derived from those master steps.

Prerequisites

Before you follow the example, make sure you have the following items:

• Samples added to the system.

• A custom field named Actual Equipment Start Time that has been configured on a master step (master step field).

• On the master step , you have configured the field to display on the Record Details milestone, in the Master Step Fields section.

Code Example

Detailed information for each step run in Clarity LIMS, including its name, LIMS ID, and custom fields can be viewed on the Record Details screen.

In the image below, an Actual Equipment Start Time master step field has been configured to display in the Step Details section of the Record Details screen. However, a value for this field has not yet been specified.

Step 1. Request the Process Resource

Before you can change the value of a process UDF/custom field, you must first request the individual process resource via a GET HTTP call. The XML returned from a GET on the individual process resource contains the information about that process. The following GET method provides the complete XML structure for the process:

The variable processNode now holds the complete XML structure retrieved from the resource at processURI.

The variable startTimeUDF references the XML element node contained in the structure that relates to the Actual Equipment Start Time UDF/custom field (if one exists).

The variable newStartTime is a string initialized with a value from the method parseInstrumentStartTimes. The details of this method are omitted from this example, but its function is to parse the date and time the instrument started from a log file.

The following code shows the XML structure for the process, as stored in the variable processNode. There are no child UDF/custom field nodes.

Step 2. Update the Process Resource

After modifying the process stored in the variable processNode, you can use a PUT method to update the process resource.

You can check if the UDF/custom field exists by verifying the value of the startTimeUDF variable. If the value is not null, then the field is defined and you can set a new value in the XML. If the field does not exist, you will must append a new node to the process XML resource using the UDF/custom field name and new value.

Before you can append a node to the XML, you must first specify the namespace for the new node. You can use the Groovy built-in QName class to do this. A QName object defines the qualified name of an XML element and specifies its namespace. The node you are specifying is a UDF element, so the namespace is http://genologics.com/ri/userdefined. The local part is field and the prefix is udf for the QName, which specifies the element as a UDF/custom field.

To append a new node to the process using the appendNode method of the variable processNode (which appends a node with the specified QName, attributes, and value). Specify the following attributes for the UDF/custom field element:

• the type

• the name

Both of these elements must match a UDF/custom field that has been specified in the Configuration window for the process type.

• The variable processNode now holds the complete XML structure for the process with the updated, or added, UDF named Actual Equipment Start Time.

You can save the changes you have made to the process using a PUT method on the process resource:

The PUT updates the sample resource at the specified URI using the complete XML representation, including the new value for Actual Instrument Start Time.

If the PUT was successful, it returns the XML resource, as shown below. The updated information is also available at the http://yourIPaddress/api/v2/processes/A22-BMJ-100927-24-2188\ URI.

• If the PUT was unsuccessful, an XML resource is returned with contents that detail why the call was unsuccessful. In the following example error, an incorrect UDF/custom field name was specified. A UDF/custom field named Equipment Start Time was created in the process resource, but no UDF/custom field with that name was configured for the process type/master step.\

Expected Output and Results

The Step Details section of the updated Record Details screen now shows the Actual Equipment Start Time value.

The ability to modify process properties allows you to update automatically and store lab activity information as it becomes available. Information from equipment log files or other data sources can be collected in this way.

Updating per-run or per-process/step information is powerful because the information can be used to optimize lab work (eg, by tracking trends over time). The data can be compared by instrument, length of run, lab conditions, and even against quality of molecular results.

Attachments

UpdateProcessUDFInfo.groovy:

When samples are processed in the lab, they are sometimes re-arrayed in complex ways that are pre-defined.

You can use the REST API and automation functionality that will allow a user to initiate a step that:

1. Uses a file to define a re-array pattern

2. Executes the step using that re-array pattern. Since the pattern is pre-defined, this will decrease the likelihood of an error in recording the re-array.

Pooling steps require that each input analyte artifact (derived sample) in the step be inserted into a pool. You can automate this task by using the API steps pooling endpoint. Automation of pooling allows you to reduce error and user interaction with Clarity LIMS.

In this example, a script pools samples based on the value of the pool id user-defined field (UDF)/custom field of the artifact.

As of Clarity LIMS v5, the term user-defined field (UDF) has been replaced with custom field in the user interface. However, the API resource is still called UDF.

• Master step fields—Configured on master steps. Master step fields only apply to the following: