When working with files, you can do the following:
Many lab processes create small files to summarize results. This example attaches a file to the LIMS server file storage repository, rather than linking to an existing file on the network. Linking to an existing file is covered in Attach Files Located Outside the Default File Storage Repository. Both examples are useful in practice, depending on your network, storage architecture, and the size of the file.
The file attachment method used in this example is equivalent to a lab scientist manually importing a file and attaching it to a file placeholder in the LIMS user interface.
Before POSTing to the files resource, you must make sure that the file exists in the location referenced by the content-location element. If the file does not exist in this location, the POST fails.
You can also use EPP/automation to handle attaching files. Though this method is not as flexible, it does attach a file automatically. With this method, the attached files are copied to the Clarity LIMS server and attached based on a LIMS ID in the created file name. Automation scripts are powerful, but they are only called when the process/step is created, whereas the code in this example can run at any time.
Before you follow the example, make sure that you have the following items:
Samples that have been added to the system.
A process with analytes (derived samples) as inputs and result files as outputs that has been run on some samples.
The file to be attached (example uses results.csv) that exists in the same directory from which the script is executed.
The JSch library that has been imported onto the classpath. JSch is used by some Groovy closures (refer to fileSFTPHelper) in the attached script for SFTP logic. For more information, refer to http://www.jcraft.com/jsch/
After you run a process with a result file output, a file placeholder icon displays next to the result file.
Attaching a file to a process output requires the use of glsstorage and files resources.
glsstorage assigns a file location, much like a file placeholder in the user interface. The glsstorage POST is a request to create a unique file name (name and directory) for a future disk file. The files resource is used to associate the physical disk location to the ResultFile artifact. In combination, these resources allow flexible management of files, and the integration of external file manipulation and transfer tools.
You can attach a file as follows.
Create a storage location for the file, using a POST to glsstorage. This returns the XML needed to POST to the files resource. The XML includes a content-location.
Copy the file to the new storage location.
POST the file XML to attach the file to the result file placeholder.
The first step in attaching a file to the placeholder is to create a location for the file on the LIMS server, using a POST to the glsstorage resource.
A POST to glsstorage requires the attached-to and original-location child elements to be defined within the file XML content. (These are the only two elements required to post to glsstorage.)
The file XML content is created using Groovy's StreamingMarkupBuilder.
Our example code begins by defining the current location of the file in the variable fileOriginalLocation.
In addition, the artifactURI variable is defined as the URI with which to associate the file. In this case, that resource is a ResultFile artifact.
Before the POST to glsstorage, the previously created XML appears as follows:
The following code shows the POST to glsstorage. The XML returned by the POST to glsstorage is stored in the variable resolvedFileNode.
The XML returned by the POST method includes a new child element – content-location. The content-location is the new directory and file name on the LIMS server to which the file should be copied.
If the POST to glsstorage was unsuccessful, an XML document explaining the error is returned. For example, if the artifact specified by artifactURI does not exist, then the POST will fail, and resolvedFileNode will hold the content shown below:
The next step is to copy the file from current to new location. SFTP is used through the fileSFTPHelper closure.
If there has never been a file attached to the ResultFile placeholder, the directory specified by the content-location element will not exist.
The fileSFTPHelper Groovy closure takes care of creating the directory required, using the destRemoteFileURI parameter. It then copies the file to the new file location using SFTP.
To make sure the copy was successful, you can check that the $sftpSuccess variable was 'true'.
The glsstorage methods do not create, move, or delete disk files. Use file operations that fit within your scripting or system automation practices.
After you SFTP the file to the content-location, you can POST resolvedFileNode to the files (list) REST resource.
POSTs to this resource require file XML content with the attached-to, original-location, and content-location child elements defined.
After the POST to the files resource, the file information is attached to the resource specified in the attached-to child element:
The POST to the files (list) REST resource associates a physical file with the ResultFile artifact. In the user interface, this POST changes the icon from a placeholder to an attached file.
If the POST was successful, the updated XML is in the returnNode variable, and contains a LIMS ID and a URI, as shown in the following example:
The artifact resource for the result file now contains a file element with the file LIMS ID and URI:
When the script completes, an attached file icon will display in the LIMS client (Operations Interface shown here). The lab scientist can download the file from this view.
cookbookExamples.properties:
PostFileToProcess.groovy:
This example attaches a file to the Clarity LIMS server file storage repository instead of linking to an existing file on the network. For more information on linking to an existing file, refer to is covered in .
Both examples are useful in practice, depending on your network, storage architecture, and the size of the file. The file attachment method used in this example is equivalent to a lab scientist manually importing a file and attaching it to a file placeholder in the Clarity LIMS user interface.
Before you follow the example, make sure you have the following items:
Files can be attached to projects, steps or result file artifacts.
This script uses the python package Requests and relies upon glsapiutil.py.
A compatible version of API (v2).
You can also use EPP/automation to handle attaching files. Though this method is not as flexible, it does attach a file automatically. With this method, the attached files are copied to the Clarity LIMS server and attached based on a LIMS ID in the created file name. Automation scripts are powerful, but they are only called when the process/step is created, whereas the code in this example can run at any time. For more information, see .
Attaching a file requires the use of glsstorage and files resources.
The glsstorage resource assigns a file location, much like a file placeholder in the user interface. The files resource is used to associate the physical disk location to the ResultFile artifact. In combination, these resources allow flexible management of files, and the integration of external file manipulation and transfer tools.
Attaching a file is done in three main steps:
Using a POST, create a storage location for the file to glsstorage. This returns the XML that POSTs to the files resource. The XML includes a content-location.
Link the file to the placeholder (creating a unique file LIMS ID) by POSTing the glsstorage response to the /api/v2/files endpoint.
Using the /api/v2/files/{limsid}/upload endpoint, upload the file.
replace file APPLICATION EXAMPLE.py:
In Next Generation Sequencing (NGS), a large amount of data is generated in a single run. Because the volume of data is so large, it makes sense to link to data files as they exist on the network, rather than copying files to the Clarity LIMS file server.
This example shows how you can use the files resource to associate a process output result file with a file on the network using the HTTP protocol.
Before you follow the example, make sure you have the following items:
Samples that have been added to the system.
A NGS process that takes analyte (derived sample) inputs and creates a single result file output for each input.
A flow cell container with 8 rows and 1 column. Both the rows and columns are numbers and start at 1.
Samples that have been added to a flow cell and have run the flow cell through your NGS process.
An HTTP file store that has been set up in the Clarity LIMS configuration file.
The appropriate directory structure and files to be linked to your HTTP file store. For this example, you need a directory with the same name as the flow cell on which you run your process.
You require a subdirectory for each lane named by lane number (eg, 1 containing the file you want to link).
The name of the file must be easily programmatically determined (eg, s_1_export.txt for lane number 1).
A compatible version of API v2.
This example assumes you are using the standard non-production scripting sandbox server, which uses Apache to serve files with HTTP. For more information on this server, see Useful Tools.
If you plan to use an alternative file storage configuration, contact the IT administrator of the Clarity LIMS server.
The administrator uses the omxprops-ConfigTool.jar configuration tool. For more information on this configuration tool, refer to Non-Production Scripting Sandbox Server - IT Admin Guide.
The omxprops-ConfigTool.jar tool is at /opt/gls/clarity/tools/propertytool/. For more information, see Remote HTTP Filestore Setup.
After running a NGS process on a full flow cell, in the Operations Interface, the Input/Output Explorer for the process run shows the relationship between the inputs in the flow cell and the result file output placeholders.
Associating a file that resides on a server that is accessible via the HTTP protocol requires the following steps:
Matching up the file with the correct file placeholder.
Constructing the XML to POST to the files resource.
A POST to the files resource, which associates the file and creates the link between the artifact and the file.
The following code snippet shows how, by starting with only the name of the flow cell, we can obtain the API resource for the flow cell container and the NGS process run.
Because the location of all the files we want to link is known, the combination of the flow cell and NGS process contains all the additional information required to link each file to the correct file placeholder, so that the results are associated with the correct sample.
Get a containers list filtering by container name, where the name is equal to the name of the directory. Your container name must be unique in the system as only one result is expected by this script.
Get the full XML for the container. The container list only contains the URI and LIMS ID of the container and we require all the placements as well.
Using the LIMS ID of one of the placements in the container, use the API to find the processes that have used it as an input. For this example, the expectation is that only one process has been run on the flow cell. If that is not the case, the first one returned is used.
Complete the XML with the input-output-map and make another call to the API to retrieve the complete process. As for the container case, the list resource just gives us the URI and LIMS ID of the process.
The code in this example includes some helper closures and a persistent single HTTP connection. The persistent connection helps improve the performance of the script.
The location of the file to be linked is constructed for each POST, based on the directory structure. For each input-output-map element in the process it is stored in the FileURI variable. The value provided here is used in both the <content-location> and <original-location> elements in the file node that will be posted.
The location of the result file artifact is stored in the variable outputURI. It is obtained from the <output> element in the input-output-map:
The first requirement is to create an XML resource for the file you want to submit to the system.
As in the example from Attach a File to a File Placeholder with REST, you create the resource using StreamingMarkupBuilder, which you use in a POST to the files resource.
When you construct the individual file XML, you must specify the content-location, attach-to, and original-location child nodes.
In this example, you are posting a file link to a file in an HTTP file storage location, so the content-location and original-location are the same.
The POST requires an XML node as input. Therefore, you must convert fileDoc from a writable closure to an XML node using GLSGeneusRestApiUtils.xmlStringToNode.
You can then submit the new file resource to the API via a POST.
The following example shows the XML for FileNode, which is used in the POST:
The POST command does the following:
POSTs the XML constructed by StreamingMarkupBuilder to the files resource.
Adds a link from the files list resource to the new file.
After the POST executes, a link to the new file resource is added to the ResultFile artifact resource – because it was specified in the attached-to field of the new file resource.
The following code is the XML for the ResultFile artifact after the POST, which contains a link to the new file resource on the last line:
The file resource available from the API is slightly altered from the XML submitted to the POST. The file is assigned a LIMS ID and a URI.
The following code is the XML resource for the file that is available from the API:
The POST to the files resource associates a physical file to the ResultFile artifact. In the user interface, this POST changes the file icon.
Before POSTing to the files resource, you must make sure that the file exists in the location referenced by the content-location element. If the file does not exist in this location, the POST fails.
The value returned from the POST is stored in the returnNode variable.
If the command was successful, the above XML is returned. If the command was unsuccessful, an XML document explaining the error is returned. For example, if a file was already attached to the result file at artifactURI, the following document is stored in returnNode, as returned from the POST. In this case, the file that is already attached to the artifact must first be removed via the file resource DELETE method.
After a successful POST HTTP command, in the Operations Interface, the process summary view's Input/Output Explorer shows all the attached files.
AttachingFileNotOnClarity.groovy:
