Data Management
Added information to indicate that the action to link data is now only possible from the root level of your project. Previously, this was possible on all levels, but the data was always linked on root level which was not in line with user expectations
Improved description of linked folders, now indicating that linking in linked folder and creating subfolders in linked folders is not possible
Added note explaining that there is a discrepancy between the API specification and the actual response for the GET /api/projects/{projectId}/analyses/{analysisId}/outputs endpoint because analysisData is retained for backward compatibility
Clarified that adding a "New folder name" when moving data to another folder does NOT rename a folder. Instead, it creates a new folder to move data to. You can rename a folder by moving files to a new folder, then deleting the original contents
Added information about which AWS storage classes are supported and how they map to statuses for ICA data
Added note to advise users to use Intelligent tiering without the Archive and Deep Archive tiers, as these tiers are currently not supported
If data is modified in the source or destination folder of a Move job before the Move job is complete, this will lead to the job aborting. Added documentation for how to fix this issue if it occurs
Added information that move jobs will fail if any files being moved are in the "Partial" or "Archived" state. For copy jobs, files in the "Partial" or "Archived" state will be skipped
Flow
Added information about JSON based pipeline input forms
Improved descriptions of scratch space
Documented possible causes of exit code 55 for analysis failure (external event such as spot termination, node draining, ...)
Bench
Updated Bench workspace documentation to indicate the storage size can be between 1GB and 16TB
Updated information on bring your own bench image software volumes
General
Transferred cloud-analysis auto-launch content to Illumina Connected Software help site
Updated CLI release history with new build
Data Management
Added information that "Has access + archive" should not be used as permission in IAM
Updated storage settings for Indonesia, Israel, and UAE regions
Added troubleshooting information on "Unable to generate credentials from the objectstore as the requested path is too long."
Documented that storage credential actions are now integrated with the grid view
Added information on how dynamic linking works. This replaces static linking
Added warning that when you move data from one location to another, you should not change the source data while the Move job is in progress, as this will result in jobs getting stuck in progress
Flow
Fixed broken link on Basic Nextflow Tutorial page
Bench
Added explanation on how to use bring-your-own-bench-image functionality
Added information on how to open Jupyter notebook workspaces in a new tab
General
General updates and improvements
Transferred cross-product content, such as cloud-analysis auto-launch, to Illumina Connected Software help site
Updated the link for software registration from ICA to the cross-product ICS software registration page
Replaced all instances of "volume configuration" with "storage configuration" for consistency, as they have the same definition
Created a new page in the "Reference" section for Document Revision History, separating it from Software Release Notes
Data Management
Added information on how the Service Connector determines when files can be transferred to prevent partial files from being processed
Documented the new 100 concurrent file upload limit on GUI-side upload
CLI
New version of the CLI documentation
Notifications
Corrected configuration explanation error in the configure notifications section for Amazon SQS
Flow
Added information on how to resubmit an analysis with modified values
Added information that setting the Nextflow XML to multiValue="true" will result in ICA treating it as a list
Documented the limitation of tags. The limit is 500 characters per tag and no more than 250 tags per tag type
Sorting on duration is not supported and has been removed from the GUI as duration is dynamically calculated when it is requested
Documented how the user is now able to write a pipeline which executes files located in the /data/bin directory of the runtime environment
Updated a screenshot in the CWL Dragen pipeline creation section
Documented the new files tab on the pipeline screen which provides an easier overview of files related to your pipeline
Base
Added an example query for how to obtain the costs incurred by the individual steps of an analysis
Bench
Corrected file path in Jupyter Notebooks section
Added hint that using the option "latest" for public docker images increases the likelihood of errors
Improved explanation of how workspace permissions work
Cohorts
Updated Cohorts information
Storage
Added information on storage credentials and how to configure them
Added a video to demonstrate how to connect an AWS S3 bucket with SSE-KMS Encryption enabled
Network Settings
Updated network setting URLs
Added network settings for Israel region
Workgroups
Documented that there is now an "all workgroup" option to easily reset workgroup filtering
Added a hint that your BaseSpace workgroup needs access to ICA if you want to use BSSH-managed projects
General
Added tutorial in ICS docs for how to manually launch pipelines on NovaSeq X data by linking data from externally-managed project to ICA projects
Data Management
Added information on allowed search field characters when searching for projects or data
Added a note to clarify that data is always linked to the root folder of the project
Documented that single files can now be directly downloaded without the need to open them first
API
Added guidance on how to prevent excessive API requests from exceeding the ICA API rate limit
Added a description to the API guide of how to copy multiple files with an API request
CLI
Added a warning to not use the CP -f command to copy or move data to a mounted location, as this will result in data loss
Updated CLI Command Index
Notifications
The data unarchived event ICA_DATA_114 has been added to the list of notification codes
Flow
Documented how the user is now able to write a pipeline which executes files located in the /data/bin directory of the runtime environment
General
General updates and improvements
Added mention of the collapsible menu bar in a brief overview of the GUI
Moved content in the Troubleshooting Guide from References section to relevant sections in the docs
Added a tutorial to demonstrate how to create and launch a Nextflow pipeline using the ICA CLI
Data Management
Added information on copying Docker images and reference data to different regions
Added hint that the service connector or CLI should be used for uploading large files
Added information about how bundles and projects need to be in the same region in order to be linked
Documented how there will no longer be a status displayed when adding an external docker image, as the status would always be shown as active even when the external image could not be reached
Added data formats: adat, zarr, tiff and wsi to the list of supported file types
API
Added information on how to set API keys
Corrected the example for cursor-based pagination, which was using offset-based parameters
CLI
Updated command index with new commands
Updated the CLI release history with new build
Notifications
Added how to create SNS notifications in a region different from where the control plane is installed
Flow
Corrected the statement that, for CWL pipelines, creating a scratch space has to be done as part to the requirement section instead of a hint
Included how the analysis-details URL now points to the correct information when sharing the URL from your browser with other users
Added information for three new transfer compute types: transfer-small, transfer-medium, and transfer-large
Documented system capacities to reflect that there is a maximum of 50 concurrent runs per tenant
Bench
Updated URL whitelist expression for restricted bench workspaces
Updated Bench reference docs with details about when workspace will be stopped if not accessed for long duration
Cohorts
Added information on sharing a Cohort as a Bundle
Storage
Added guidance on using key prefixes when setting up an S3 storage configuration
Added a list of common issues encountered when connecting an AWS S3 bucket through a storage configuration to the Troubleshooting Guide
General
Added video demo for Oncology Walkthrough guided example
Added video demo for how to connect an AWS S3 bucket to ICA
Removed obsolete Instrument Run Upload tutorial, now replaced with Sequencer Integration tutorial
Updated sequencer integration guide
Improved the description for “Tenant” billing mode
Data Management
Added information about how to create a new service connector from the “Schedule for Download” screen
Added information about the character limit for links in bundles and pipelines
CLI
Documented the new CLI environment variable ICA_NO_RETRY_RATE_LIMITING, which disables the retry mechanism
Updated the CLI release history with new build
Flow
Added information on how ICA compute type will be determined based on CPU and Memory
Bench
Updated documentation to reflect how paths can be added to allowed URLs on restricted setting for Bench
General
General updates and improvements
Updated the table of contents to match the Connected Analytics sidebar
Updated region named "Europe" to "Germany" for consistency with other regions named by country
Updated sequencer reference guide to reflect support for DRAGEN Amplicon autolaunch analysis using the NextSeq 1000/2000
Added link to ICA YouTube channel which contains video demos to Resources page
Data Management
Documented how folders cannot include a backslash in their name
Documented how you can copy data between folders within the same project
API
Added information on how to start a pipeline with the API to the ICA API Beginner Guide
When using the Sample Creation Batch API endpoint to link nested files with the same name, the duplicate file on the root level will be ignored instead of resulting in an error. The API documentation has been updated to reflect this.
CLI
Updated the CLI release history with new build
Flow
Documented how additional files related to your Nextflow or CWL pipeline will be displayed alphabetically
Updated the CWL CLI Workflow tutorial to include a description of the behavior for how `runtime.ram` and `runtime.cpu` are evaluated
Updated analysis lifecycle documentation to include all analysis statuses
Base
Updated the Base Tables documentation to reflect how defining the precision of numbers when creating tables with SQL is not supported as this will result in a rounding issue
Updated Base Tables documentation to reflect how you can now see the Load Status of files uploaded to the table
Updated the Base Query documentation to inform users how to find the correct capitalization to use when querying the ICA reference data, which is case sensitive
Bench
Updated documentation to reflect how paths can be added to allowed URLs on restricted settings for Bench
Storage
Updated the AWS S3 help guide to include information about S3 bucket policies
Revised the Storage Configuration Troubleshooting Guide to be a general Troubleshooting Guide and included a section for data transfer issues
Network Settings
Updated the "Network Settings" reference page with URLs for Indonesia and UAE
General
General updates and improvements to reflect changes in graphical user interface
Added a tutorial to demonstrate the end-to-end user flow for planning a NovaSeq X run with BaseSpace Run Planner and performing the run with data storage and automated analysis in ICA
Updated the list of auto-launch analysis workflows in the Sequencer Reference table
Data Management
Updated the list of addresses which need to be whitelisted for the service connector
Notifications
Updated the list of Notification Subscriptions to include all possible notifications
CLI
Updated the CLI Command index to reflect addition of --exclude-source-path flag for ‘project data download’ command
Flow
Added information on how to add scratch space to all processes in a pipeline
Added note stating that batch jobs which have a final state such as Failed or Succeeded are removed from the activity list after 7 days
For Nextflow version 20.10.10, using the ‘copy’ method in thepublishDir directive for uploading output files that consume large amounts of storage may cause workflow runs to complete with missing files. Added information on workarounds to prevent this from occurring.
Bench
Added the 500GB limit per file for the FUSE driver
General
General updates and improvements
Clarified that bundle assets can only be linked once to the same data set
Removed outdated information about creating new tabs from workspace description
Added information about stopping Workspaces and removed information about pausing Workspaces, as pausing is no longer an option
Added information about creating an OAuth access token
Clarified that upload and download rights act independent of user roles
Updated the table which shows the regions ICA is hosted on to include all the available regions in alphabetical order
Added language specifying that folders with the name '.' are not supported
API
Added preliminary beginner's tutorial for ICA API usage
Added information about how to use data stored in Basespace as an input when starting an analysis via the API
CLI
Updated the CLI release history
Flow
Added information on how to define scratch space in CWL workflows
Described analysis runtime environment used to execute user-defined pipeline tasks, steps, and processes
Updated the Nextflow DRAGEN tutorial to account for no default scratch space
Base
Added explanation of how to query arrays in table data
Added information about how to query views from Data Catalogues using the Snowflake FLATTEN function
Storage
Made it more explicit that the custom bucket name needs to be entered in the IAM configuration
Updated instructions for creating an AWS IAM user policy to remove a permission which is not correct
General
General updates and improvements
Updated organization of Table of Contents
API
Documented how to archive or delete files programmatically using API endpoints
Added information about how the sort parameter is used to choose the default pagination
Flow
Added guidance about the ica_logs subfolder in the analysis output folder
Updated default scratch space reference documentation
Cohorts
Updated documentation to reflect OMOP ingestion and UI search for Procedures. The documentation changes also reflect instruction on the simplified ingestion and comparison sharing
Storage
Added explanation of the impact of billing mode when users bring their own S3 storage
General
General updates and improvements
Sequencer Integration: Added information that auto-launch from instrument is not currently compatible with MiSeq systems
Data Management
Documented an alternate method of adding files to samples and explained how to remove files from samples
Provided more guidance on data transfer options
Updated reference documentation for Data Copy functionality
Added a message to inform users that folder copy can be impacted in cases where a folder with the same name already exists in the target project
Added reference documentation for data integrity recommended practices
CLI
Updated CLI release history
Notifications
Updated Filtering section of Notifications page
Flow
Described analysis runtime environment used to execute user-defined pipeline tasks, steps, and processes
Updated ‘CWL: Scatter-Gather’ tutorial
Added documentation for analysis timeout
Base
Added a new tutorial for creating a Python dashboard to access Base metrics via API
Added information about Snowflake user/role setup
Included sample TSV file in Base Basics tutorial
Bench
Updated Fuse Driver troubleshooting
General
General updates and improvement
Added link to Illumina Knowledge resource
Updated analysis autolaunch reference docs
Updated the NovaSeq 6000 analysis auto-launch tutorial
SDK: Included new reference to Illumina's ICA Python SDK resource available on GitHub.com
Data Management
Add documentation around storage configuration synchronization behavior
Documented the new Data Copy functionality
Provided more guidance on data transfer options
Added a section describing Data Catalogue functionality
Added guidance about not using file names beginning with '/'
Service Connector: Added troubleshooting guidance for service connector
API
Added tip for users experiencing API rate limiting
Added troubleshooting guide for GetTemporaryUploadCredentialsAsync failure error scenario
CLI
Updated the CLI release history page with all required build files
Base
Improved descriptions and scenarios of Base tables and queries
Cohorts
Documented Cohorts Diagnosis Concept attribute search box functionality
General
General updates and improvements
Added ICA Terminology page to Reference
Logging: Documented event logging transfer of ownership
Data Management
Document how to change Project owner
Updated Storage Size table for Compute on Pricing page to include XL, 2XL, 3XL
Improved explanation of hiding Projects when connecting to your own S3 bucket for data storage
Added default and shared explanation to storage documentation
Updated documentation to reflect the temp credentials returned for AWS datastore is for 36 hours
CLI
Added documentation for CLI config settings
Flow
Documented event for Analysis status transition
Added Nextflow best practices to reduce Analysis failures
Updated Pipeline activation code behavior explanation
Added documentation for specifying external docker images (i.e. Docker Hub)
Base
Added a tutorial for using Base via the CLI
Updated documentation to mention that Schema does not contain type ‘variant’
Documented Base query which joins to tables can result in duplicate columns
Documented SnowSQL show view vs show grants
Bench
Documented Bench permission changes
Cohorts
RNASeq ingestion
Applicable user-supplied RNASeq visualizations – Raw Expression Values in GTEx-like table
NeedlePlot
Structure Variant list
General
General improvements and fixes
User Management: Added details about how users can manage an overview of their Projects
Included a new reference to Illumina’s ICA Python SDK resource available on Github
Data Management
Clarified the visibility of Batch jobs for different users in a Tenant
Bundles: Provided more details about Bundle linking and unlinking with data in folders
CLI
Updated the CLI command index
Updated the CLI release history page with all required build files
Added link to legacy ICA v1 CLI release history page
Flow
Described Analysis runtime environment used to execute user-defined pipeline tasks, steps, and processes
Added an explanation for how external S3 files can be used as inputs to Pipelines
Bench
Added an explanation that Project/Tenant Administrators can stop running Workspaces
Added guidance for downloading data when using JupyterLab Notebooks
Cohorts
Added instructions for RNA-Seq data ingestion into Cohorts
Added reference documentation for multi-cohort marker frequency and needle plot comparisons
Network Settings
Updated the list of whitelisted URLs on https://help.ica.illumina.com/reference/r-networksettings
General
General updates and improvements
User Management: Improved explanation of Team functionality
Data Management
Added guidance on versioning Bring Your Own storage buckets
Improved explanation on linking to data from other projects
Described functionality of Secondary Data
Updated guidance for file upload size
Updated configuration guidance for Bring Your Own Storage bucket using S3
Updated list of supported Data formats
Added a tutorial for mounting project data using the CLI
API
Added information to launch analysis API spec on output mappings overwrite behavior when files already exist on the target project during analysis output
CLI
Added CLI authentication best practices for setting up, validating, and resetting the configuration of a user's profile
Network Settings
Updated Network Settings regional S3 URLs list
Flow
Added a tutorial for building and running a CWL graphical pipeline
Included best practices for minimizing errors in Nextflow pipelines by adding retry logic
Base
Updated the Base schedule descriptions to include information about Metadata and Administrative schedules, as well about frequency of scheduled ingestions
Bench
Updated restrictions of available features when in the FUSE mount in a Workspace
Updated descriptions about Bench restrictions, including details about whitelisted URLs
General
General updates and improvements
Updated guidance for selecting Project Location and availability of Storage Bundles when configuring a new Project
Updated details about paging in the Event Log
Refined and expanded guidance for sequencer integration and analysis auto-launch, including adding a new tutorial for NovaSeq 6000 and expanding the sequencer reference table
Added tutorial for using scatter/gather for Nextflow pipeline
Updated information about data archive pricing
Updated software registration guide
Data Management
Added guidance that a sample must be complete to be linked to a Bundle
Included information about receiving and using Entitled Bundles
CLI
Updated CLI command index
Updated CLI binary and checksum files
Flow
Refined Pipelines documentation around clone function
Base
Added information that Base Tables must not contain reserved Snowflake terms in their definition
Added description for Copy Number Variants and Structural Variants tables now viewable in ICA Base for Cohorts
Cohorts
Added description for hg19 support as an alternative during ingestion for Cohorts
General
General updates and improvements
Refined and expanded reference documents for sequencer integration and analysis autolaunch, including new pages for sequencer reference and troubleshooting
Added tutorial for setting up a notification service and Lambda function in AWS for chaining pipelines
Included guidance for setting override behavior for file output of advanced output mappings
Described selection of entitlements when creating a Project
Updated FAQ with guidance on analysis run times
Updated explanation for adding Terms of Use to a Bundle
Included details for the Event Log
Data Management
Explained how to unlink a folder from a project
CLI
Updated CLI command index
Fixed a bug in the CLI release history files and updated integrity check instructions
Flow
Described limitation on amount of input files for analyses
Base
Added tutorial for accessing Base databases and tables via snowSQL
Refined and expanded description of Base Activity screen
Bench
Expanded description of Bench Activity screen
Explained multi-factor authentication compatibility with Bench and provide workaround suggestion
Cohorts
Added instructions to remove specific subjects from a cohort
Documented cohort needle plot enhancements: List of subjects and specific frequency data entry
Updated cohorts ingestion instruction including metadata documents, GWAS ingestion, OMOP ingestion, Structural Variant ingestion, and hg19 lift-over information
Added reference documentation for using Cohorts attributes metadata template
Added instruction for cohorts attribute query enhancement on numeric data
Storage
Clarified details about S3 bucket behavior and Bring Your Own Storage bucket (BYOB) compatibility
Network Settings
Updated documentation about network settings for pipelines
Provided guidance about deleting old versions of Bring Your Own Storage buckets
General
Added reference documentation for deep links to a folder
Updated Nextflow DRAGEN tutorial to include details about how to obtain access to the docker image on ICA
Improved documentation about pipeline activation codes
Explained limit of number of projects and bundles allowed per tenant
Data Management
Created new page for and updated list of available data file formats
Updated guidance for requirements for adding assets to a bundle
Updated service connector reference docs
Explained progress details for service connector
Added information about archiving or unarchiving a file
API
Added descriptions of cursor and offset pagination when using the API
CLI
Expanded explanation of default settings for the CLI
Base
Added guidance for Base JSON export
Updated supported formats for Base table/query export
Updated description of Base Activity screen
Added guidance for Base schedule's optional parameters
Bench
Provided documentation for available Bench compute types
Added documentation for Analysis auto launch
General
Updated reference docs for workflow session events
Provided details about price-per-sample and zero-cost pipelines
Described pipeline Proprietary flag available
Data Management
Service Connector: Updated information in the service connector screen to reflect the name change from "Type of Files" to the more accurate "Assign Format"
Added details about which data formats are compatible with UI Preview
Added details about syntax for linking to individual analyses and workflow sessions
Explained how to add assets to a bundle
CLI
Provided a command index for the ICA v2 CLI
Expanded explanation on default settings for the CLI
Clarified details about hidden metadata files behavior during CLI copy for MacOS users
Flow
Added information about standard and economy tier selection for compute types
Expanded guidance for XML-based input form required of pipelines defined using the "Code" mode
Expanded guidance on importing external docker images in ICA when running a pipeline
Included information about tile/grid view for analysis logs
Added guidance about balancing analysis queuing priorities when setting up a Project
Updated information about Batch Jobs
Added guidance about overriding CWL workflow requirements
Base
Added guidance on using Cohorts data in ICA Base
Cohorts
Included video overview of Cohorts Multiomics Cancer workflow
Updated steps for Cohorts ingestion
Updated steps for Cohorts analysis
General
Added guidance about modifying existing Tools already added to a Pipeline
Updated tutorial for building a DRAGEN pipeline using Nextflow
Notifications
Updated list of system event notifications
Flow
Clarified expected behavior and recommended order of steps when connecting input to Tools and editing Input Details when building graphical CWL pipelines
Added information about details for monitoring Batch Jobs in Project Activity Screen
Added Pipeline “tips and tricks"
Storage
Added guidance on expected behavior when configuring user-managed AWS Buckets with Projects
General
Added tutorial for using scatter/gather for CWL graphical pipelines
Added information about Documentation tab in Nextflow pipelines
Updated links and fixed typos
Data Management
Provided list of ICA-supported data file formats
API
Included guidance on creating custom events for projects via the API
Data Management
Added data privacy recommendations regarding uploading data files
Added guidance on supported characters in file/folder names
Added documentation for linking bundles to a project
Cohorts
Added ICA Cohorts Module documentation, including summary of features and example walk-throughs
Storage
Added instructions for setting up ICA v2 Bring Your Own Bucket (BYOB) with SSE-KMS key
Updated bucket name character length constraints
General
Added Quick Registration Guide
Added documentation for metadata models
Added tutorial for creating/running CWL pipeline on CLI
Added end-to-end tutorial highlighting DRAGEN Germline pipeline
Data Management
Added documentation for reference data
Flow
Added documentation for analysis output mappings
Added documentation for Nextflow configuration