Troubleshooting Failures

This troubleshooting section outlines the commonly encountered errors while using the DRAGEN StrataMap analysis software and how to address them.

Core (previously called ICA) specific troubleshooting

Locating the Key Log File

If you encounter a problem while using the DRAGEN StrataMap analysis software, the best and first log file to check is called ICA_nextflow_issues.log. This is also the file the Illumina support team may request to help troubleshoot the problem.

The ICA_nextflow_issues.log file can be found in the top-level directory of the Output files for each run. It can be accessed from the Output files section of the Core interface in the bottom right panel of the analysis run page.

This log often contains human-readable error messages indicating the problem with the pipeline. A few commonly encountered issues are explained below:

DAM specific troubleshooting

Common Issues and Solutions

License Issues

Error: "DRAGEN license does not include StrataMap feature"

Solution:

1. Check your license status:

   dragen_lic

2. Verify the "Spatial" section shows "Active" status
3. Contact Illumina support if license is missing or expired

Missing Files

Error: "File does not exist: /path/to/file"

Solution:

1. Verify the file path is correct and absolute
2. Check file permissions:

   ls -la /path/to/file

3. Ensure the file is accessible from the DRAGEN server

Genome Not Found

Error: "Genome directory does not exist"

Solution:

1. Verify genome resources are installed:

   dragen-app-manager resource list

2. Re-run installation if needed:

   ./install_DRAGEN_StrataMap.sh

Pipeline Failures

Error: Pipeline fails during execution

Solution:

1. Check the pipeline trace file:

   cat outputs/intermediate_results/logs/full_pipeline_trace.txt

2. Look for tasks with non-COMPLETED status
3. Check individual task logs in outputs/intermediate_results/logs directory
4. Review nextflow.log for detailed error messages
5. Review DAM_startup.log for errors related to DRAGEN-App-Manager file mounts and argument validation.

Pipeline Issues (Core or DAM)

1. Run Failed: Wrong Slide ID

Error message in in ICA_nextflow_issues.log:

ICA_S3_Download (1)' terminated with an error exit status (1)

Explanation:

The pipeline tried to locate the coordiate map, but could not because there was a typo in the slide ID. Re-start analysis with the correct slide ID. When kicking off a manual launch in Core, the 'slide ID' field in the Core user interface (UI) will override what is in the sample sheet, so the sample sheet does not need to be edited before re-running analysis.

2. Run Failed: Wrong Slide ID and BCL Pair

Error message in ICA_nextflow_issues.log: Only X% of the reads are in samples. The BCL folder does not appear to match the slide ID

Explanation:

The pipeline will spatially locate the reads from the BCL folder. The process will fail if it cannot spatially locate a majority of the reads, as this indicates a mismatch between the slide ID and BCL folder. To fix this issue:

1. Confirm that the slide ID you entered while using the Illumina StrataMap Image Tool exactly matches the number on the slide. There may be a typo, or two slides may have been mixed up.
2. Re-launch analysis manually using the Core user interface (UI) and input the slide ID. The values input in the UI will override the sample sheet, so the incorrect slide ID in the sample sheet does not need to be changed, but it may be better to update it as well for future reference. The slide ID is listed next to each sample in the Cloud_StrataMap_Data section of the sample sheet.

3. Run Failed: Incorrect Sample ID and OME-TIFFs

Possible error messages in ICA_nextflow_issues.log: Found more than 1 ome-tiff file for sample: SAMPLEID

No such file or directory: [Path] SampleID_registered.ome.tiff

Explanation:

The pipeline expects one OME-TIFF per SampleID with the correct naming structure. To fix this issue:

1. If it errors out after finding >1 or 0 OME-TIFFs for a given SampleID, confirm that Core shows the correct number of OME-TIFFs and that their SampleIDs match the uploaded OME-TIFFs.

4. Run Finished but There is no HTML and Missing Results for Some Samples

It is possible that results are missing for some of your samples even though the pipeline "succeeded" in Core. The pipeline is designed so that it attempts to analyze every sample individually, so if one sample fails, the pipeline will still "succeed" if it successfully completes for the rest of the samples. There are two known ways that one sample from an analysis can fail:

4a: Random Cell Segmentation Failure

Error messages in ICA_nextflow_issues.log:

Missing output file(s) 06_cell_segmentation/[SAMPLEID]/cell_segmentation/cell_border*/*_cell_Segmentation_mask.tif expected by process `getCellSegmentationOutputChannel

and

Process aggregateStats:makehtml (1) terminated with an error exit status (1)

Explanation:

Occasionally, we have seen one sample from an analysis fail during the cell segmentation process, and only that sample is missing tertiary results. Please re-run the analysis and it should complete successfully.

4b: Insufficient Memory

ICA

Possible error messages in ICA_nextflow_issues.log: .command.sh: line X killed

error 143

Command exit status: 137

insufficient memory

and

Process aggregateStats:makehtml (1) terminated with an error exit status (1)

Explanation:

To reduce costs, the pipeline dynamically allocates computing resources based on tissue area and flow cell type. On occasion, the pipeline will allocate too few resources and will crash if a process requires more than the allocated memory. To fix this issue:

1. Determine the resource configuration of the run that failed. This can be determined by looking at the Nextflow timeline (tab on top bar within ICA) and then locating the name of the TertiaryFlow process, which will be either ICA_TertiaryFlow_M, ICA_TertiaryFlow_L, or ICA_TertiaryFlow_XL.
2. Re-run the pipeline by manually launching it, and instead of selecting Dynamic, pick a resource one size up from the failed run (for example, if the run failed with Medium (M), select Large; if it failed with Large, select XLarge (XL)). If it failed with XLarge, please reach out to Illumina support.

DAM

Review outputs/intermediate_results/logs/full_pipeline_trace.txt Example:

• Search through the "exit" column for entries 134 or 137
• If this is the case, call run_DRAGEN_StrataMap _x.x.x.x.sh with "--resource-size XL"
• If the analysis still fails then contact customer support.