Illumina SupportIllumina Knowledge

Attach a File to a File Placeholder with REST

Many lab processes create small files to summarize results. This example attaches a file to the LIMS server file storage repository, rather than linking to an existing file on the network. Linking to an existing file is covered in Attach Files Located Outside the Default File Storage Repository. Both examples are useful in practice, depending on your network, storage architecture, and the size of the file.

The file attachment method used in this example is equivalent to a lab scientist manually importing a file and attaching it to a file placeholder in the LIMS user interface.

Before POSTing to the files resource, you must make sure that the file exists in the location referenced by the content-location element. If the file does not exist in this location, the POST fails.

Using EPP / Automation to Attach Files

You can also use EPP/automation to handle attaching files. Though this method is not as flexible, it does attach a file automatically. With this method, the attached files are copied to the Clarity LIMS server and attached based on a LIMS ID in the created file name. Automation scripts are powerful, but they are only called when the process/step is created, whereas the code in this example can run at any time.

Prerequisites

Before you follow the example, make sure that you have the following items:

  • Samples that have been added to the system.

  • A process with analytes (derived samples) as inputs and result files as outputs that has been run on some samples.

  • The file to be attached (example uses results.csv) that exists in the same directory from which the script is executed.

  • The JSch library that has been imported onto the classpath. JSch is used by some Groovy closures (refer to fileSFTPHelper) in the attached script for SFTP logic. For more information, refer to http://www.jcraft.com/jsch/

Code example

After you run a process with a result file output, a file placeholder icon displays next to the result file.

Attaching a file to a process output requires the use of glsstorage and files resources.

glsstorage assigns a file location, much like a file placeholder in the user interface. The glsstorage POST is a request to create a unique file name (name and directory) for a future disk file. The files resource is used to associate the physical disk location to the ResultFile artifact. In combination, these resources allow flexible management of files, and the integration of external file manipulation and transfer tools.

You can attach a file as follows.

  1. Create a storage location for the file, using a POST to glsstorage. This returns the XML needed to POST to the files resource. The XML includes a content-location.

  2. Copy the file to the new storage location.

  3. POST the file XML to attach the file to the result file placeholder.

Step 1. Create a Storage Location for the File

The first step in attaching a file to the placeholder is to create a location for the file on the LIMS server, using a POST to the glsstorage resource.

A POST to glsstorage requires the attached-to and original-location child elements to be defined within the file XML content. (These are the only two elements required to post to glsstorage.)

The file XML content is created using Groovy's StreamingMarkupBuilder.

Our example code begins by defining the current location of the file in the variable fileOriginalLocation.

In addition, the artifactURI variable is defined as the URI with which to associate the file. In this case, that resource is a ResultFile artifact.

// Create a new file Node using StreamingMarkupBuilder
def fileDoc = new StreamingMarkupBuilder().bind {
  mkp.xmlDeclaration()
  mkp.declareNamespace(file:'http://genologics.com/ri/file')
  'file:file'{
    'attached-to'("${artifactURI}")
    'original-location'("${fileOriginalLocation}")
  }
}

Before the POST to glsstorage, the previously created XML appears as follows:

<file:file xmlns:file="http://genologics.com/ri/file">
  <attached-to>http://yourIPaddress/api/v2/artifacts/AFF853A40AP2</attached-to>
  <original-location>/home/glsftp/Testing/results.csv</original-location>
</file:file>

The following code shows the POST to glsstorage. The XML returned by the POST to glsstorage is stored in the variable resolvedFileNode.

// Post the file Node to the API
unresolvedFileNode = GLSRestApiUtils.xmlStringToNode(fileDoc.toString())
resolvedFileNode = GLSRestApiUtils.httpPOST(unresolvedFileNode, "${apiIndexURI}/glsstorage", username, password)
println GLSRestApiUtils.nodeToXmlString(resolvedFileNode)

The XML returned by the POST method includes a new child element – content-location. The content-location is the new directory and file name on the LIMS server to which the file should be copied.

<file:file xmlns:file="http://genologics.com/ri/file">
  <content-location>sftp://yourIPaddress/home/glsftp/Process/2010/9/A54-BMJ-100921-24-2178/AFF853A40AP2-40-2911.csv</content-location>
  <attached-to>http://yourIPaddress/api/v2/artifacts/AFF853A40AP2</attached-to>
  <original-location>/home/glsftp/Testing/results.csv</original-location>
</file:file>

If the POST to glsstorage was unsuccessful, an XML document explaining the error is returned. For example, if the artifact specified by artifactURI does not exist, then the POST will fail, and resolvedFileNode will hold the content shown below:

<exc:exception xmlns:exc="http://genologics.com/ri/exception">
  <message>Requested artifact not found</message>
</exc:exception>

Step 2. Copy the File to the New Storage Location

The next step is to copy the file from current to new location. SFTP is used through the fileSFTPHelper closure.

// Copy the file from its current location
def fileFinalLocation = new URI(resolvedFileNode.'content-location'[0].text())
File sourceFileObj = new File(fileOriginalLocation)
def sftpSuccess = fileSFTPHelper(fileFinalLocation, sourceFileObj, glsftpUsername, glsftpPassword)
println("SFTP success : $sftpSuccess")

If there has never been a file attached to the ResultFile placeholder, the directory specified by the content-location element will not exist.

The fileSFTPHelper Groovy closure takes care of creating the directory required, using the destRemoteFileURI parameter. It then copies the file to the new file location using SFTP.

To make sure the copy was successful, you can check that the $sftpSuccess variable was 'true'.

The glsstorage methods do not create, move, or delete disk files. Use file operations that fit within your scripting or system automation practices.

Step 3. Attach File to the Result File Placeholder

After you SFTP the file to the content-location, you can POST resolvedFileNode to the files (list) REST resource.

POSTs to this resource require file XML content with the attached-to, original-location, and content-location child elements defined.

After the POST to the files resource, the file information is attached to the resource specified in the attached-to child element:

// Post the file node to the API
returnNode = GLSRestApiUtils.httpPOST(resolvedFileNode, "${apiIndexURI}/files", username, password)
println GLSRestApiUtils.nodeToXmlString(returnNode) 

The POST to the files (list) REST resource associates a physical file with the ResultFile artifact. In the user interface, this POST changes the icon from a placeholder to an attached file.

If the POST was successful, the updated XML is in the returnNode variable, and contains a LIMS ID and a URI, as shown in the following example:

<file:file uri="http://yourIPaddress/api/v2/files/AFF853A40AP2-40-2908" limsid="AFF853A40AP2-40-2908">
    <content-location>sftp://yourIPaddress/home/glsftp/Process/2010/9/A54-BMJ-100921-24-2178/AFF853A40AP2-40-2907.csv</content-location>
    <attached-to>http://yourIPaddress/api/v2/artifacts/AFF853A40AP2</attached-to>
    <original-location>/home/glsftp/Testing/results.csv</original-location>
    <is-published>false</is-published>
</file:file>

The artifact resource for the result file now contains a file element with the file LIMS ID and URI:

<art:artifact uri="http://yourIPaddress/api/v2/artifacts/AFF853A40AP2?state=20968" limsid="AFF853A40AP2">
    <name>Calcaneus-1</name>
    <type>ResultFile</type>
    <output-type>ResultFile</output-type>
    <parent-process uri="http://yourIPaddress/api/v2/processes/A54-BMJ-100921-24-2178" limsid="A54-BMJ-100921-24-2178"/>
    <qc-flag>UNKNOWN</qc-flag>
    <sample uri="http://yourIPaddress/api/v2/samples/AFF853A40" limsid="AFF853A40"/>
    <file:file limsid="AFF853A40AP2-40-2908" uri="http://yourIPaddress/api/v2/files/AFF853A40AP2-40-2908"/>
</art:artifact> 

Expected Output and Results

When the script completes, an attached file icon will display in the LIMS client (Operations Interface shown here). The lab scientist can download the file from this view.

Attachments

cookbookExamples.properties:

PostFileToProcess.groovy:

PreviousAttach Files Located Outside the Default File Storage RepositoryNextWork with Controls

Last updated