Nextflow
ICA supports running pipelines defined using Nextflow. See this tutorial for an example.
In order to run Nextflow pipelines, the following process-level attributes within the Nextflow definition must be considered.
System Information
Nextflow version
20.10.0 (deprecated ⚠️), 22.04.3 (supported ✅), 24.10.2 (default ⭐)
Executor
Kubernetes
The following table shows when which Nextflow version is
default (⭐) This version will be proposed when creating a new Nextflow pipeline.
supported (✅) This version can be selected when you do not want the default Nextflow version.
deprecated (⚠️) This version can not be selected for new pipelines, but pipelines using this version will still work.
removed (❌). This version can not be selected when creating new pipelines and pipelines using this version will no longer work.
The switchover always happens in the January release of that year.
v20.10.0
⚠️
❌
❌
❌
v22.04.3
✅
⚠️
❌
❌
v24.10.2
⭐
⭐
✅
✅
v25.10.x
✅
⭐
✅
v26.10.x
✅
⭐
v27.10.x
✅
Nextflow Version
You can select the Nextflow version while building a pipeline as follows:
GUI
Select the Nextflow version at Projects > your_project > flow > pipelines > your_pipeline > Details tab.
API
Select the Nextflow version by setting it in the optional field "pipelineLanguageVersionId
".
When not set, a default Nextflow version will be used for the pipeline.
Compute Node
For each compute type, you can choose between the scheduler.illumina.com/lifecycle: standard
(default - AWS on-demand) or scheduler.illumina.com/lifecycle: economy
(AWS spot instance) tiers.
Compute Type
To specify a compute type for a Nextflow process, use the pod
directive within each process. Set the annotation
to scheduler.illumina.com/presetSize
and the value
to the desired compute type. A list of available compute types can be found here. The default compute type, when this directive is not specified, is standard-small
(2 CPUs and 8 GB of memory).
pod annotation: 'scheduler.illumina.com/presetSize', value: 'fpga2-medium'
Inputs
Inputs are specified via the XML input form or JSON-based input form. The specified code
in the XML will correspond to the field in the params
object that is available in the workflow. Refer to the tutorial for an example.
Outputs
Outputs for Nextflow pipelines are uploaded from the out
folder in the attached shared filesystem. The publishDir
directive can be used to symlink (recommended), copy or move data to the correct folder. Data will be uploaded to the ICA project after the pipeline execution completes.
publishDir 'out', mode: 'symlink'
Nextflow Configuration
During execution, the Nextflow pipeline runner determines the environment settings based on values passed via the command-line or via a configuration file (see Nextflow Configuration documentation). When creating a Nextflow pipeline, use the nextflow.config tab in the UI (or API) to specify a nextflow configuration file to be used when launching the pipeline.
Syntax highlighting is determined by the file type, but you can select alternative syntax highlighting with the drop-down selection list.

The following configuration settings will be ignored if provided as they are overridden by the system:
executor.name
executor.queueSize
k8s.namespace
k8s.serviceAccount
k8s.launchDir
k8s.projectDir
k8s.workDir
k8s.storageClaimName
k8s.storageMountPath
trace.enabled
trace.file
trace.fields
timeline.enabled
timeline.file
report.enabled
report.file
dag.enabled
dag.file
Best Practices
Process Time
Setting a timeout to between 2 and 4 times the expected processing time with the time directive for processes or task will ensure that no stuck processes remain indefinitely. Stuck process keep incurring costs for the occupied resources, so if the process can not complete within that timespan, it is safer and more economical to end the process and retry.
Sample Sheet File Ingestion
When you want to use a sample sheet with references to files as Nextflow input, add an extra input to the pipeline. This extra input lets the user select the samplesheet-mentioned files from their project. At run time, those files will get staged in the working directory, and when Nextflow parses the samplesheet and looks for those files without paths, it will find them there. You can not use file paths in a sample sheet without selecting the files in the input form because files are only passed as file/folder ids in the API payload when the analysis is launched.
You can include public data such as http urls because Nextflow is able download those. Nextflow is also able to download publicly accessible S3 urls (s3://...). You can not use Illumina's urn:ilmn:ica:region:... structure.
Last updated
Was this helpful?