Illumina SupportIllumina Knowledge

API Beginner Guide

API Basics

Any operation from the ICA graphical user interface can also be performed with the API.

The following are some basic examples on how to use the API. These examples are based on using Python as programming language. For other languages, please see their native documentation on API usage.

Prerequisites

  • An installed copy of Python. (https://www.python.org/)

  • The package installer for python (pip) (https://pip.pypa.io/)

  • Having the python requests library installed (pip install requests)

Authenticating

One of the easiest authentication methods is by means of API keys. To generate an API key, refer to the Get Started section. This key is then used in your Python code to authenticate the API calls. It is best practice to regularly update your API keys.

API keys are valid for a single user, so any information you request is for that user to which the key belongs. For this reason, it is best practice to create a dedicated API user so you can manage the access rights for the API by managing those user rights.

API Reference

There is a dedicated API Reference where you can enter your API key and try out the different API commands and get an overview of the available parameters.

Converting curl to Python

The examples on the API Reference page use curl (Client URL) while Python uses Python requests. There are a number of online tools to automatically convert from curl to python.

To get the curl command,

  1. Look up the endpoint you want to use on the API reference page.

  2. Select Try it out.

  3. Enter the necessary parameters.

  4. Select Execute.

  5. Copy the resulting curl command.

Never paste your API authentication key into online tools when performing curl conversion as this poses a significant security risk.

In the most basic form, the curl command

curl my.curlcommand.com

becomes

import requests
response = requests.get('http://my.curlcommand.com')

You will see the following options in the curl commands on the API Reference page.

-H means header.

-X means the string is passed "as is" without interpretation.

curl -X 'GET' 'https://my.curlcommand.com' -H 'HeaderName: HeaderValue'

becomes

import requests
headers = {
    'HeaderName': 'HeaderValue',
}
response = requests.get('https://my.curlcommand.com', headers=headers)

Simple API Examples

Request a list of event codes

This is a straightforward request without parameters which can be used to to verify your connection.

The API call is

response = requests.get('https://ica.illumina.com/ica/rest/api/eventcodes', headers={'X-API-Key': '<your_generated_API_key>'})

In this example, the API key is part of your API call, which means you must update all API calls when the key changes. A better practice is to put this API key in the headers so it is easier to maintain. The full code then becomes

# The requests library will allow you to make HTTP requests.
import requests

# Replace <your_generated_API_key> with your actual generated API key here.
headers = {
    'X-API-Key': '<your_generated_API_key>',
}

# Store the API request in response.
response = requests.get('https://ica.illumina.com/ica/rest/api/eventCodes', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Response status code: ", response.status_code)

# Display the data from the request.
print(response.json())

Pretty-printing the result

The list of application codes was returned as a single line, which makes it difficult to read, so let's pretty-print the result.

# The requests library will allow you to make HTTP requests.
import requests

# JSON will allow us to format and interpret the output.
import json

# Replace <your_generated_API_key> with your actual generated API key here.
headers = {
    'X-API-Key': '<your_generated_API_key>',
}

# Store the API request in response.
response = requests.get('https://ica.illumina.com/ica/rest/api/eventCodes', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Print JSON data in readable format with indentation and sorting.
print(json.dumps(My_API_Data, indent=3, sort_keys=True))

Retrieving a list of projects

Now that we are able to retrieve information with the API, we can use it for a more practical request like retrieving a list of projects. This API request can also take parameters.

Retrieve all projects

First, we pass the request without parameters to retrieve all projects.

# The requests library will allow you to make HTTP requests.
import requests

# JSON will allow us to format and interpret the output.
import json

# Replace <your_generated_API_key> with your actual generated API key here.
headers = {
    'X-API-Key': '<your_generated_API_key>',
}

# Store the API request in response.
response = requests.get('https://ica.illumina.com/ica/rest/api/projects', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Print JSON data in readable format with indentation and sorting.
print(json.dumps(My_API_Data, indent=3, sort_keys=True))

Single parameter

The easiest way to pass a parameter is by appending it to the API request. The following API request will list the projects with a filter on CAT as user tag.

response = requests.get('https://ica.illumina.com/ica/rest/api/projects?userTags=CAT', headers=headers)

Multiple parameters

If you only want entries that have both the tags CAT and WOLF, you would append them like this:

response = requests.get('https://ica.illumina.com/ica/rest/api/projects?userTags=CAT&userTags=WOLF', headers=headers)

Copying Data

To copy data, you need to know:

  • Your generated API key.

  • The dataId of the files and folders which you want to copy (their syntax is fil.hexadecimal_identifier and fol.hexadecimal_identifier). You can select a file or folder in the GUI and select it to see the Id (Projects > your_project > Data > your_file > Data details > Id) or you can use the /api/projects/{projectId}/data endpoint.

  • The destination project to which you want to copy the data.

  • The destination folder within the destination project to which you want to copy the data (fol.hexadecimal_identifier).

  • What to do when the destination files or folders already exist (OVERWRITE, SKIP or RENAME).

The full code will then be as follows:

# The requests library will allow you to make HTTP requests.
import requests

# Fill out your generated API key.
headers = {
    'accept': 'application/vnd.illumina.v3+json',
    'X-API-Key': '<your_generated_API_key>',
    'Content-Type': 'application/vnd.illumina.v3+json',
}

# Enter the files and folders, the destination folder, and the action to perform when the destination data already exists.
data = '{"items": [{"dataId": "fil.0123456789abcdef"}, {"dataId": "fil.735040537abcdef"}], "destinationFolderId": "fol.1234567890abcdef", "copyUserTags": true,"copyTechnicalTags": true,"copyInstrumentInfo": true,"actionOnExist": "SKIP"}'

# Replace <Project_Identifier> with the actual identifier of the destination project.
response = requests.post(
    'https://ica.illumina.com/ica/rest/api/projects/**<Project_Identifier>**/dataCopyBatch',
    headers=headers,
    data=data,
)

# Display the response status code.
print("Response status code: ", response.status_code)

Combined API Example - Running a Pipeline

Now that we have done individual API requests, we can combine them and use the output of one request as input for the next request. When you want to run a pipeline, you need a number of input parameters. In order to obtain these parameters, you need to make a number of API calls first and use the returned results as part of your request to run the pipeline. In the examples below, we will build up the requests one by one so you can run them individually first to see how they work. These examples only follow the happy path to keep them as simple as possible. If you program them for a full project, remember to add error handling. You can also use the GUI to get all the parameters or write them down after performing the individual API calls in this section. Then, you can build your final API call with those values fixed.

Initialization

This block must be added at the beginning of your code

# The requests library will allow you to make HTTP requests.
import requests

# JSON will allow us to format and interpret the output.
import json

# Replace <your_generated_API_key> with your actual generated API key here.
headers = {
    'X-API-Key': '<your_generated_API_key>',
}

Look for a project in the list of Projects

Previously, we already requested a list of all projects, now we add a search parameter to look for a project called MyProject. (Replace MyProject with the name of the project you want to look for).

# Store the API request in response. Here we look for a project called "MyProject".
response = requests.get('https://ica.illumina.com/ica/rest/api/projects?search=MyProject', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Print JSON data in readable format with indentation and sorting.
print(json.dumps(My_API_Data, indent=3, sort_keys=True))

Now that we have found our project by name, we need to get the unique project id, which we will use in the combined requests. To get the id, we add the following line to the end of the code above.

print(My_API_Data['items'][0]['id'])

Syntax ['items'][0]['id'] means we look for the items list, 0 means we take the first entry (as we presume our filter was accurate enough to only return the correct result and we don't have duplicate project names) and id means we take the data from the id field. Similarly, you can build other expressions to give you the data you want to see, such as ['items'][0]['urn'] to get the urn or ['items'][0]['tags']['userTags'] to get the list of user tags.

Once we have the identifier we need, we add it to a variable which we will call Project_Identifier in our examples.

# Get the project identifier.
Project_Identifier = My_API_Data['items'][0]['id']

Retrieve the Pipelines of your Project

Once we have the identifier of our project, we can fill it out in the request to list the pipelines which are part of our project.

response = requests.get('https://ica.illumina.com/ica/rest/api/projects/'+(Project_Identifier)+'/pipelines?', headers=headers)

This will give us all the available pipelines for that project. As we will only want to run a single pipeline, we can search for our pipeline, which in this example will be the basic_pipeline. Unfortunately, this API call has no direct search parameter, so when we get the list of pipelines, we will look for the id and store that in a variable which we will call Pipeline_Identifier in our examples as follows:

# Find Pipeline
# Store the API request in response. Here we look for the list of pipelines in MyProject.
response = requests.get('https://ica.illumina.com/ica/rest/api/projects/'+(Project_Identifier)+'/pipelines', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Find Pipeline Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Store the list of pipelines for further processing.
pipelineslist = json.dumps(My_API_Data)

# Set "basic_pipeline" as the pipeline to search for. Replace this with your target pipeline.
target_pipeline = "basic_pipeline"
found_pipeline = None

# Look for the code to match basic_pipeline and store the ID.
for item in My_API_Data['items']:
    if 'pipeline' in item and item['pipeline'].get('code') == target_pipeline:
        found_pipeline = item['pipeline']
        Pipeline_Identifier = found_pipeline['id']
        break
print("Pipeline Identifier: " + Pipeline_Identifier)

Find which parameters the Pipeline needs.

Once we know the project identifier and the pipeline identifier, we can create an API request to retrieve the list of input parameters which are needed for the pipeline. We will consider a simple pipeline which only needs a file as input. If your pipeline has more input parameters, you will need to set those as well.

# Find Parameters
# Store the API request in response. Here we look for the Parameters in basic_pipeline
response = requests.get('https://ica.illumina.com/ica/rest/api/pipelines/'+(Pipeline_Identifier)+'/inputParameters', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Find Parameters Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Get the parameters and store in the Parameters variable.
Parameters = My_API_Data['items'][0]['code']
print("Parameters: ",Parameters)

Find the Storage Size to use for the analysis.

Here we will look for the id of the extra small storage size. This is done with the 0 in the My_API_Data['items'][0]['id']

# Store the API request in response. Here we look for the analysis storage size.
response = requests.get('https://ica.illumina.com/ica/rest/api/analysisStorages', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Find analysisStorages Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Get the storage size. We will select extra small.
Storage_Size = My_API_Data['items'][0]['id']
print("Storage_Size: ",Storage_Size)

Find the files to use as input for your pipeline.

Now we will look for a file "testExample" which we want to use as input and store the file id.

# Get Input File
# Store the API request in response. Here we look for the Files testExample.
response = requests.get('https://ica.illumina.com/ica/rest/api/projects/'+(Project_Identifier)+'/data?fullText=testExample', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Find input file Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data
My_API_Data = response.json()

# Get the first file ID.
InputFile = My_API_Data['items'][0]['data']['id']
print("InputFile id: ",InputFile)

Start the Pipeline.

Finally, we can run the analysis with parameters filled out.

Postheaders = {
    'accept': 'application/vnd.illumina.v4+json',
    'X-API-Key': '<your_generated_API_key>',
    'Content-Type': 'application/vnd.illumina.v4+json',
}

data = '{"userReference":"api_example","pipelineId":"'+(Pipeline_Identifier)+'","analysisStorageId":"'+(Storage_Size)+'","analysisInput":{"inputs":[{"parameterCode":"'+(Parameters)+'","dataIds":["'+(InputFile)+'"]}]}}'

response = requests.post(
    'https://ica.illumina.com/ica/rest/api/projects/'+(Project_Identifier)+'/analysis:nextflow',headers=Postheaders,data=data,
)

Complete code example

# The requests library will allow you to make HTTP requests.
import requests

# JSON will allow us to format and interpret the output.
import json

# Replace <your_generated_API_key> with your actual generated API key here.
headers = {
    'X-API-Key': '<your_generated_API_key>',
}

# Replace <your_generated_API_key> with your actual generated API key here.
Postheaders = {
    'accept': 'application/vnd.illumina.v4+json',
    'X-API-Key': '<your_generated_API_key>',
    'Content-Type': 'application/vnd.illumina.v4+json',
}

# Find project
# Store the API request in response. Here we look for a project called "MyProject".
response = requests.get('https://ica.illumina.com/ica/rest/api/projects?search=MyProject', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Find Project response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Get the project identifier.
Project_Identifier = My_API_Data['items'][0]['id']
print("Project_Identifier: ",Project_Identifier)

# Find Pipeline
# Store the API request in response. Here we look for the list of pipelines in MyProject.
response = requests.get('https://ica.illumina.com/ica/rest/api/projects/'+(Project_Identifier)+'/pipelines', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Find Pipeline Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Store the list of pipelines for further processing.
pipelineslist = json.dumps(My_API_Data)

# Set "basic_pipeline" as the pipeline to search for. Replace this with your target pipeline.
target_pipeline = "basic_pipeline"
found_pipeline = None

# Look for the code to match basic_pipeline and store the ID.
for item in My_API_Data['items']:
    if 'pipeline' in item and item['pipeline'].get('code') == target_pipeline:
        found_pipeline = item['pipeline']
        Pipeline_Identifier = found_pipeline['id']
        break
print("Pipeline Identifier: " + Pipeline_Identifier)

# Find Parameters
# Store the API request in response. Here we look for the Parameters in basic_pipeline.
response = requests.get('https://ica.illumina.com/ica/rest/api/pipelines/'+(Pipeline_Identifier)+'/inputParameters', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Find Parameters Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Get the parameters and store in the Parameters variable.
Parameters = My_API_Data['items'][0]['code']
print("Parameters: ",Parameters)

# Get Storage Size
# Store the API request in response. Here we look for the analysis storage size.
response = requests.get('https://ica.illumina.com/ica/rest/api/analysisStorages', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Find analysisStorages Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Get the storage size. We will select extra small.
Storage_Size = My_API_Data['items'][0]['id']
print("Storage_Size: ",Storage_Size)

# Get Input File
# Store the API request in response. Here we look for the Files testExample.
response = requests.get('https://ica.illumina.com/ica/rest/api/projects/'+(Project_Identifier)+'/data?fullText=testExample', headers=headers)

# Display the response status code. Code 200 means the request succeeded.
print("Find input file Response status code: ", response.status_code)

# Put the JSON data from the response in My_API_Data.
My_API_Data = response.json()

# Get the first file ID.
InputFile = My_API_Data['items'][0]['data']['id']
print("InputFile id: ",InputFile)

# Finally, we can run the analysis with parameters filled out.
data = '{"userReference":"api_example","pipelineId":"'+(Pipeline_Identifier)+'","tags":{"technicalTags":[],"userTags":[],"referenceTags":[]},"analysisStorageId":"'+(Storage_Size)+'","analysisInput":{"inputs":[{"parameterCode":"'+(Parameters)+'","dataIds":["'+(InputFile)+'"]}]}}'
print (data)
response = requests.post('https://ica.illumina.com/ica/rest/api/projects/'+(Project_Identifier)+'/analysis:nextflow',headers=Postheaders,data=data,)
print("Post Response status code: ", response.status_code)
PreviousBench ICA Python LibraryNextLaunch Pipelines on CLI

Last updated