2022

2022 December 13 - ICA v2.11.0

Features and Enhancements

• New notifications added for data upload, data created, and analyses with ‘aborted’ state

• Flow: Progress on batch jobs can now be monitored in the Activity screen under the ‘Batch Jobs’ tab

• Base: Shared database activity is available in the Base Activity Logging Screen

Cohorts:

• Enhanced Attribute Queries: Filter logic now allows users to include or exclude filter criteria.

• Ingestion Wizard: Human readable project name added to email notification of ingestion status

• Ingestion Wizard: Notification email now includes the project name and job failure details

• Ingestion Wizard: File lists are now sortable and filterable by type

• Ingestion Wizard: UI enhancements on ingestion wizard selections

• Ingestion Wizard: Based on data type chosen, the list of files for ingestion will be filtered for easier navigation of files to ingest

• Share Cohorts with Collaborators - Cohort definitions are now sharable in a project

• Statistical significance on Cohort Analysis - A p-value calculation is added to the Survival Comparison data points

Fixed Issues

• Resolved an error when creating a Bucket Name with more than 63 characters

• Fixed an issue causing failure when attempting to link "complex" folders (i.e., contains 100 sub-folders each with sub-folders) via GUI throws a “…transaction is not active…" error

• Fixed an issue causing intermittent failure of "complex" folder (i.e., contains 100 sub-folders each with sub-folders) downloads

• Fixed an issue so that the API to create data download URL correctly returns a 503 instead of a 404 response when the underlying system is experiencing network-related issues

• Fixed an issue where the user is not redirected to the Project overview view upon clicking ‘Hide a Project’

• Fixed an issue which prevented linking and unlinking multiple Bundles to and from a Project in a single operation via the GUI

• Fixed an issue to show records in Bundle Data Transfers when data from a Bundle is accessed via View Tab and when downloaded

• Fixed an issue showing stack trace when attempting to add Secondary Data to Data when not allowed

• Fixed issue where the File Upload selection disappears after navigating back to Data after navigating to the Data History Tab

• Fixed an issue where create folder session and get upload URL APIs are allowed for data accessible via an entitled Bundle

• Fixed an issue where OpenAPI spec for delete project bundle API does not include proper "accept" header when using "Try It" functionality

• Fixed an issue where write-access can be acquired for data accessible via an entitled Bundle

• Fixed an issue where Project owner of an externally managed Project could not open Analysis Overview

• Fixed an issue where multiple OS versions of the Service Connector are incorrectly downloaded at once when a single version is selected

• Fixed an issue when selecting the "empty" option in the Operating System drop-down when downloading a connector

• Fixed an issue where users with Contributor role and no data upload access can add data to samples without error

• Added an event log entry for Service Connector log uploads initiated by Illumina Support

• Flow: Fixed an issue to support Aborting an Analysis immediately after starting it

• Flow: Fixed an issue when publishDir in Nextflow pipelines contains symlinks to data, those data are not included when uploading analysis outputs to a Project

• Flow: Fixed an issue preventing closure of tabs in the Logs section for technical steps in the Analysis Details view

• Flow: Cancel button added to the define CWL Pipeline selection dialogue window, and the order of buttons has been changed

• Fixed an issue causing pipelines to stall for hours after all steps are completed

• Flow: Fixed an issue where Nextflow pipelines fail with ‘pod 404 not found’ error

• Base: Linked files from a Bundle can be inserted in a Base table

• Base: Fixed an error on previewing Base tables that have been shared from a Bundle

• Base: Fixed an issue where Base API to retrieve a list of Base jobs is not paging correctly

• Bench: Fixed an issue where Bench Workspaces could not be opened with a double-click

• Bench: After the “Save and Start” of a new Bench Workspace, the breadcrumb shows the actual name of the Workspace

• Bench: Clarified error messages when creating new Workspaces with file permissions

• Cohorts: Issue fixed where cohort is created during filtering process for RNA Expression on Apply

• Cohorts: Issue fixed where order of items in the Survival Curve comparison legend are shuffled when reloaded

• Cohorts: Issue fixed where long comparison names caused by a string concatenation between cohorts were causing the comparison feature to malfunction

• Cohorts: Issue fixed where comparison name updates not reflected in the UI

• Cohorts: Issue fixed where option still remains checked after Molecular filter removed from the right panel while on the project tab

• Cohorts: Issue fixed where TCGA data not showing Molecular breakdown for selected genes

• Cohorts: Issue fixed where data selections in the correlation feature where the second option visually shows half of the result in the UI

• Cohorts: Issue fixed where "missense_and_ptv_all" missing from PHWAS/GWAS variant types

2022 November 8 - ICA v2.10.0

Features and Enhancements

• Added format auto-detection for VCF (.hard-filtered.vcf.gz, .cnv.vcf.gz) and TAR (.v8.tar, .tar.gz) files

• Pipeline documentation will be available if present when starting a new Analysis

• Added Workflow Session event topics

• Base: Shared databases are visible in Base tables. Shared databases are read-only and queryable tables originating from ICA modules such as Cohorts

• Reduced pipeline spin up time delays due to shared files system provisioning

• Added support for Illumina applications to add custom attributes to data, samples, workflow sessions, and analyses

• API: "creatorId" for Project data records returned in the API

• API: Added instructions for paging parameters on list endpoints in API specification

• API: The edit and delete API endpoint for Terms of Use are no longer available

• Updated ICA CLI to version 2.10.0

Fixed Issues

• Fixed an issue allowing unauthorized users to access external-managed Project contents via a direct link

• Fixed an issue in the Docker Repository view causing region text and check boxes to be misaligned

• Fixed an issue causing storage configuration status to incorrectly report an error

• Fixed an error preventing the creation of an AWS S3 storage configuration in a Project

• Fixed an issue where the Service Connector status incorrectly showing ‘Currently Connected’ when the associated Connector is not connected

• Fixed an intermittent error when clicking to expand a Connector rule details in the Connectivity View

• Removed red exclamation mark icon next to "Unlinked Files" tab in the Samples view; improperly indicates an error

• Fixed an issue where Bundle Terms of Use acceptance flag unset upon saving bundle version

• Fixed an issue preventing users from viewing details of an Entitled Bundle

• Fixed an issue resulting in an error message when creating Bundle Terms of Use with an existing version

• API: Fixed an issue where the API endpoint to link data to a Bundle to responds with errors when data is part of a Reference Data

• API: Fixed an issue causing 500 API error response when retrieving sample details

• Fixed an issue causing incomplete list of users when adding users within the same tenant to the Project Team

• API: Updated the API spec description for create upload URL and create inline view URL for Project Data

• Fix to prevent editing of an existing active terms of use

• Improved the Terms of Use UI flow for forcing re-acceptance upon update

• API: Added a warning response for API to create custom events when event code exceeds 50 characters

• API: Fixed the create data download URL API to correctly return a 503 instead of a 404 response when the underlying system is experiencing network-related issues

• API: Added API endpoint to hide a Project

• Fixed an issue with Boolean inputs incorrectly displayed as false on Analysis Details view (correct value is actually passed in)

• Fixed an issue where CWL parameters containing only a white space are passed incorrectly to underlying pipeline runner

• API: Fixed an issue where launch Nextflow analysis API returns a 500 error response when including duplicate parameters

• Fixed an issue causing analysis failures due to docker image authorization check timeout

• Fixed a shared storage issue causing analyses to remain In Progress until timeout

• Fixed an intermittent issue causing Analyses to be stuck in progress with inability to view logs in the UI

• Fixed an issue where Analysis output data is not immediately shown in the Analysis Details view upon completion

• Fixed an issue causing Analysis failures showing “…Connection timed out (Connection timed out)…” in stdout logs

• Fixed an issue causing CWL pipeline runner restarts during an Analysis which may lead to unexpected results

• Fixed an issue resulting in stuck analyses when invalid Docker image is specified for a task; timeout reduced to 15 minutes

• Fixed the name of Pipeline running logs tab from "workflow_runner" to "pipeline_runner" in Analysis Details view in the UI

• Fixed missing start time on Analysis tasks and incorrectly reported costs for such tasks

• Fixed an issue resulting in an error message when attempting to create a graphical CWL pipeline with a Tool with a Docker image that is not available in the Project's region

• Fixed an issue causing errors when saving a new Bench Workspace

• Fixed an issue causing out of sync Bench table and card views when a Workspace is created or deleted

• Fixed an issue causing an error when starting Bench Workspace when enabling service account on the Workspace from tenant receiving the Workspace via cross-tenant share

2022 October 17 - ICA v2.9.0

Features and Enhancements

• ICA Cohorts is released in production and added to the ICA user documentation

• Clicking on a folder in the Data view will open the folder. Once a Folder is opened, the Folder details are available on the top right of the screen.

• Added XML Configuration documentation and examples

• General ICA robustness enhancements

Fixed Issues

• Limit the Projects Overview view to 100 Projects with an indication that filtering must be used to see excluded Projects to improve UI performance

• Fixed an issue preventing removal of permissions on externally managed projects

• API – Fixed a 500 error response from Search Project Samples API when using a page token parameter

• Fixed an issue allowing Bundles to be saved without mandatory Release Version when back-navigating from the Create New Bundle screen in the graphical UI

• Fix to prevent data added to Bundle via secondary data upload on Data Details view

• API – Fixed an error message when attempting to link data to a Bundle from a Project with Data Sharing disabled

• API – Fixed an issue with tag filtering on getBundleSamples API

• Fixed an error when filtering on size in Docker Repository where size filter is ignored

• Fixed an issue where Analysis stuck in Aborting status when abort request sent during final stages of Analysis (such as Output, Upload)

• Fixed an issue where intermittent Analysis failures are caused by system scaling errors under high load

• Reduced repetitive Nextflow runner log entries from internal system logs to improve readability

• API – Fixed an error in the Launch Nextflow Pipeline API when invalid parameter names are requested

• API – Correctly support multi-value input parameters in launch Analysis APIs

• Deprecated Tools are only visible when ticking the box "Show deprecated Tools" box in both table and card views

• Fixed an issue preventing files linked to a Project from a Bundle as available to load into a Base Table

• When exporting a query result from Base the file receives the name specified by the user and contains the header from the table

• Removed unsupported AVRO and PARQUET formats from Available Formats to load data into Base tables

• Fixed an issue where Bench service account API key is not revoked when a Workspace is stopped

2022 September 12 - ICA v2.8.0 and v2.8.1

Features and Enhancements

• Added support for providing a nextflow.config file with a Nextflow Pipeline to improve pipeline building experience (see Nextflow Configuration docs)

• Each tenant is limited to 100 in-progress Analyses at any given time to prevent resource exhaustion; additional analyses will be queued. Note: For assistance with planning batch jobs which require greater than 100 concurrent analyses, please contact Illumina Tech Support.

• Added support for deep link URLs for direct navigation to child views in UI

• API – Added an API to link Data to Projects in batch

• API - Limit number of tags on data to 100; Excess tags in request will be ignored

• API - Added "slug" attribute to Pipeline, Project, and Data asset types

• API - Added support for the use of Project slug in place of ID in API request bodies

• API - Added “Uniform Resource Name” (URN) attribute to Pipelines and Project object types

• API - Added Project ID to notification payload for Pipeline Succeeded/Failed events

• API - Added owning Project ID for data records in the getAnalysisOutput API response

• API - Indicate originating Workflow/Workflow Session on Analyses returned via API, when applicable

• Added support to use bundled Pipelines and data to create Analyses via API without first linking to the Project

• Added support to automatically mount project data in Bench Workspace with ICA CLI

Fixed Issues

• Renamed “Delete” to “Manage” in the Samples page dropdown menu

• Removed the IGV button under View when selecting a VCF file

• API – Fixed an issue causing the List Files API to incorrectly respond with an error when passing a folder or file path parameter value containing a leading or training space

• Fixed an issue where files linked through a Project Connector will appear on root level as folder linking is not supported yet in the Project Connector

• Fixed an issue when deleting files from S3 along with records from ICA via either volume delete (with purgeobjectstore=true) or folder delete API calls, if there are more than 1,000 items to delete the system deleted only the first 1,000 items, however, all the data records got deleted from ICA. In this case the data in the user-managed S3 bucket and ICA was out of sync.

• Fixed an incorrect count displayed in UI when scheduling a download for a folder containing sub-folders

• Fixed an issue where scheduling a download of a folder containing sub-folders creates duplicate data transfer operations

• Disabled support for "fileNameExpression" on download rules; if provided when creating a download rule, the value will be ignored

• Fixed an issue allowing public domain users to incorrectly access ICA

• API – Fixed an issue where API to retrieve folder upload session intermittently responding incorrectly with error

• API – Fixed a 500 response from the API to add a pipeline to a bundle when pipeline has already been added

• Fixed an issue where user with "Viewer" permission for Flow in a project cannot link Samples from that Project

• Return the correct error response when creating a new Nextflow or CWL pipeline through the API or CLI when an existing pipeline with the same name already exists

• Improved UI messaging when setting up a Storage Configuration with permission error

• Fixed an issue when attempting to edit a saved incomplete Tool (i.e., missing Docker image)

• API - Fixed API to create notification channel to properly respond with error when attempting to create a duplicate notification channel

• Fixed an issue where boolean Analysis input settings are incorrectly set to false when passed to the Pipeline runner

• Fixed an issue when launching a Nextflow pipeline via API with multi-value inputs the order of the listed multi-value input is not preserved when passed to the pipeline runner

• Fixed an issue where "Re-run" analysis button is broken for Nextflow pipeline analyses

• When different versions of DRAGEN are run on the same host, reset the DRAGEN state so that the next task can correctly configure the FPGA compute instance

• Fixed an issue causing intermittent analysis failures due to system error when evaluating permission to pull Docker image

• Fixed empty start time attribute for Analysis tasks with a short duration (within a few seconds)

• Fixed an issue where analysis start time is set when in fact the analysis is still queued; analysis start time will be set when compute resources (i.e., shared file system) have been requested

• Fixed an issue where log file content for the pipeline runner is overwritten if system error occurs during an Analysis causing a pod restart; multiple log files may be created for the pipeline runner task if such an error occurs

• Fixed an issue causing UI log stream view for Analysis tasks to incorrectly show errors or blank views

• Fixed an issue causing incorrect Analysis task log to be displayed on the UI Logs view

• Fixed an issue where analyses get stuck in Running status if input file paths contain leading or trailing spaces in path for analysis stuck in running status if input files contain leading or trailing spaces

• Fixed an issue causing Analysis to be incorrectly marked as Success due to incorrect evaluation of exit code of an analysis task

• Fixed an issue preventing the same file to be passed to multiple input fields when launching an Analysis

• Workflow failures due to unavailable compute resources will fail quickly with a more descriptive error

• Fixed an issue where analysis output data is not immediately shown in the Analysis Details view after the analysis completes

• Removed the “Print” and “Save” buttons in the Analysis Report

• Fixed an issue preventing Pipeline creation and Tool editing when an invalid Tool is saved in the Tool Repository

• API – Fixed a 500 response from API to list Projects to properly return a 400 response when providing a search filter value with invalid characters

• API - Fixed a 500 response from create download URL API to properly return a 400 response when data is not in "Available" status (ie, Partial/Archived)

• Fixed an issue with loading data into a Base Table created directly in Snowflake with fields containing quotes

• Fixed an issue where data was added to Base Table twice when multiple Scheduled Jobs run simultaneously

• Fixed an issue with previewing Base Tables created directly via Snowflake

• Fixed an issue where status is stuck at "In Progress" for Base Copy Table operations

• Fixed an issue where Base Tables linked to a project from a bundle are shown as Views (read-only)

• Fixed an intermittent issue causing duplicate data in Base Tables

• API - Fixed a 500 response when sending a duplicate requests to load data for a Base Table to properly return a 400 response

• API - Fixed a 500 response from API to load data into Base Table to correctly return a 400 response when attempting to load data with a type mismatch

• API - Corrected Swagger UI responses

• Fix error "Unable to save workspace" when attempting to start a Bench Workspace

2022 July 27 - ICA v2.7.1

Fixed Issues

• Fixed an intermittent issue where analyses fail due to "Connection timed out" system error (seen in analysis error logs)

• Added retry logic to decrease rate of pipeline failures due to internal system error (e.g. errors in pipeline runner log such as connection timeout, 404 pod not found, 500 errors)

• Fixed an intermittent issue where "Cannot access folder" error is shown in pipeline error logs (most common in long-running analyses)

• Fixed an issue resulting in analysis failures due to system error under high load

• Fixed a compute instance configuration issue impacting DRAGEN analyses

2022 June 29 - ICA v2.7.0 and CLI v2.4.0

Features and Enhancements

• API - Added method to retrieve an analysis JSON input data

• Added Analysis Details views in the UI

• Added Analysis ID to the project analyses

• Support deep links for direct navigation to Analysis Details view

• Added support for multiple Nextflow versions when creating Nextflow pipeline

• API - Added method to create multiple samples in batches

• API - Improved response time performance of API that launches analysis with many data inputs

Fixed Issues

• Repaired issue where Pipeline-generated Sample Metadata is not picked up by platform

• Restored Time to delete function; now executes file deletes at the specified time

• Bench example code now lists project data and pagination correctly

• Restored ability to link secondary data to primary data

• Fixed issue where batch sample creation jobs get stuck in PARTIALLY_SUCCEEDED status

• Resolved issue where batch sample creation is not completed if data items could not be linked

• API - Fixed issue where mountPaths were ignored when launching analyses via API

• Restored function so that bundle contributors can link assets to their bundle

• Restored data linking to a sample in an ICA project

• Resolved issue where child workflow configurations prevent termination of parent workflow

• Removed incorrect error reported in cases where previous failure occurred

• Restored pipeline linking for bundle contributors

• Improved UI messaging for Storage Configuration permission errors

• API - Updated GET /api/projects/{projectId}/data to returns correct file details

2022 June 07 - ICA v2.6.0 and CLI v2.3.0

Features and Enhancements

• Re-labeled "Run" and "Runs" to “Analysis” and “Analyses” in the UI

• Allow storage configuration to be disconnected from a hidden Project

• Added support for a read-only view of Entitled Bundles and Bundles shared from another Tenant

• Changed analysis timeouts for setting up environment, downloading inputs, pipeline runner (inclusive of all child steps), and finalizing output steps to 96 hours

• API – Add idempotency key header to createAnalysis API

• API – Analysis logs are available via getAnalysisSteps API; includes streamable via websocket endpoint during execution and downloadable S3 URL upon completion

• JSON data with array field can be added in a Base table

• Cohorts – Additional submenu items now available

• CLI – Added support for setting the current Project ID to the value set in the ICA_PROJECT_UUID environment variable upon starting the CLI

• CLI – Added support for mounting Project data to local operating system (Read-Only)

• CLI – Added support for the outputParentFolderId flag for the ‘projectpripeline start’ command

• CLI – Added support for customizing the filename/path of additional Pipeline files with the pipeline create commands

• CLI – Added owning Project name/id attributes to the project data list output

• CLI – Added owning Project name attribute to output for projectdata list command

• CLI – Added commands to create and refresh API tokens (JWT)

• CLI – Added support for CLI use without writing to disk by enabling API Key and Server flags for all commands

• CLI – Added support for linking data to a Sample with projectdata upload command

Fixed Issues

• Fixed an issue with UI stability leading to slow response times

• Fixed an issue causing data uploads via browser to hang without error

• Fixed an issue with UI automatically refreshing upon load when many Projects are present in the Projects view

• Fixed an issue where Accept Share pop-up remains on screen after pressing to Accept the share

• Fixed an issue with uploading nested folders using the Windows Connector

• Fixed an issue where path separators are misinterpreted in upload rules for Windows Connectors.

• Fixed an issue where modifying a local file uploaded (tracked) using the Connector causes a Connector failure

• A User with Contributor Role for Flow and a Viewer Role for a Project can start an Analysis using input data from a Sample

• API – Fixed an issue where sampleDeepDelete API fails with 500 error response when Sample contains a linked folder in archived status

• Fixed an issue where ICA_STARTING_USER_TENANT_NAME is not set to the correct value on initial start

• API – Fixed an issue where upload rules are not deleted via the deleteUploadRule API despite a success response

• Sensitive fields in metadata models are now shown in italics

• Fixed an issue where filtering on the Data view gives incorrect results

• All Contributors in a Bundle are allowed to link assets to the Bundle

• Content of a Bundle shared via an entitlement in another Tenant is now visible

• Fixed an externally managed Project Team to only allow adding members with "Viewer" and "Allow Download" permissions

• API – Fixed an issue with project sample APIs returning 500 error response when sample is not linked to the project specified in the request

• API – Fixed issues with cursor-based paging on listData and listBundleData APIs returning expected results

• Fixed an issue where a User can create empty folders inside an externally managed Project with Viewer permission

• Fix auto-detection of AVRO and PARQUET data formats

• API – Fixed an issue where the API to link Samples to a Bundle incorrectly returns a 204 response when the Sample is not in Available status and the Sample is not actually linked

• API - Fixed an issue where changing the Project owner causes unexpected error responses from the listProjects, listProjectPermissions, and getProjectPermission APIs

• API – Fixed an issue where deleteDownloadRule API returns 204 responses but does not delete the specified download rule

• Fixed an issue causing delays in operations performed on data

• Fixed an intermittent permission error when writing to /scratch from a Pipeline

• Fix to make tags optional (not required) with Pipeline launch requests

• Fixed an issue causing Analyses to be incorrectly set to SUCCEEDED status when failure occurs

• API – Possible to create a Pipeline with a metadata model via API

• Fixed an issue where logs are not streamed during Analysis

• Fixed an issue when a User tries to start an Analysis with missing required parameters, the analysis is stuck in "awaiting input"

• Fixed an issue preventing renaming a pipeline after attempting to save and receiving error that a pipeline exists with the same name

• API – Fixed an issue when a Pipeline is launched via API with no input value provided for a nonmandatory input parameter the analysis status remains in "Awaiting Input" indefinitely

• API – Fixed an issue causing unpredictable error responses from createAnalysis API when many Analyses are launched in rapid succession

• API – Added operationID to API spec for the listAnalysisStorages and getAnalysisRawOutput APIs

• Fixed an issue when Analyses are queued and Aborted before they are Running, the status is not correctly updated in the API/UI, and they appear stuck in “Aborting”

• API – Fixed an issue with createAnalysis API causing error responses when launching many Analyses in rapid succession

• Possible to load JSON in Base with a repeated field (array)

• Tables created by the meta and admin data Schedules are updated after a sync

• Fixed an issue causing Base Table field names to be changed from lowercase to UPPERCASE during a manual sync

• Fixed an intermittent issue with Base Administrative/Meta-data Schedules creating files in project Data when schedules are run

• Fixed an intermittent issue with Base Administrative/Meta-data Schedules creating files in project Data when scheduled

• Fixed an issue where the User receives an obscure error message when creating a Tool from a Bench Workspace with an image name that already exists

• Fixed an error message upon attempting to save Bench Workspace when input values are invalid

• Fixed an issue where the Bench Workspace is temporarily inaccessible and stuck in Starting status

• CLI – Fixed an issue with list samples command output limited to 20 unique records

2022 April - ICA v2.5.0

Features and Enhancements

• The value of the "Show Hidden Projects" filter on the Project Overview page will be saved for each user

• Change analysis task log names to be more human-readable

• Analyses will queue when system load is high

• Add owning project tenant name (ICA_PROJECT_TENANT_NAME) and the tenant name of the user which last started the Workspace (ICA_STARTING_USER_TENANT_NAME) environment variables to Bench Workspaces

• Add Cohorts URL environment variable to Bench Workspaces

• API - Add support for Cross Origin Resource Sharing (CORS) to the API

Fixed Issues

• Fixed an error message in UI when attempting to link a Sample that cannot be linked

• API - Fix issue where API to update Sample metadata fields does not update the fields

• Fixed error message when creating a Bundle with a name conflict

• Fixed error message when a user attempts to view Bundle details without permission

• API - Fix Project Bundle APIs to properly link, retrieve, and unlink entitled Bundles

• Fixed to block ability to link data with Contributor role and no upload permission on the target project

• Fixed an issue where the Owner of an externally managed Project cannot add Team members with Viewer or Download permission

• Fixed field for providing an e-mail address when adding a Team member to a Project to be required

• Removed temporary directory creation by service connector

• Fix for configuring service connector to use proxy settings on Linux clients

• Fixed an issue where data downloaded through Service Connector is incomplete

• Fixed an issue causing errors in Service Connector when a directory is deleted in the UI but still present on the local machine. Data added to the folder on the local machine will not be uploaded if the corresponding folder is deleted in the UI.

• Fix to display an error message when saving a Pipeline with invalid fields

• Fix issue where CWL file/directory inputs are not mounted when provided within nested structures (ie, CommandInputRecordSchema, CommandInputArraySchema)

• Fix to show error when attempting to run a graphical CWL pipeline with an invalid definition

• Fix error when attempting to save a graphical CWL Pipeline with a Tool with an output folder and no specified format

• API - Fix to ignore input JSON field when launching graphical CWL pipelines via API

• API - Fix ignored default compute type (machine profile) defined in the Pipeline definition when launching a graphical CWL Pipeline via API

• Fixed Base Query result export file format set to UNKNOWN

• Fixed Base activity log to show Table associated with log event

• API - Ignore unknown attributes in request bodies

• API - Fixed API error responses formatted as HTML instead of JSON in error scenarios

• API - Fix list analyses API not returning the "items[].analysisStorage" field

2022 April - ICA v2.4.0, v2.4.1, v2.4.2

Features and Enhancements

• Samples may only be linked when status is Available

• Added Data ID to grid view in UI

• Added ability to select and copy tag values to the clipboard

• Added an indication of source domain in list of reference data sets when adding new data to Reference Data

• Updated the service connector client icon

• Added ability to filter projects by workgroup, including projects where user is a member of a workgroup added to the project

• Added ability to specify output directory within a project when launching a pipeline

• Added ability to Favorite Projects and filter the Projects view by Favorites

• Remove obsolete commented text from auto-generated CWL Tool definitions

• Scheduler displays the number of files to be loaded into a Base Table

• Added an indication that Base Table previews display up to 50 columns and 100 rows

• API - New endpoint to manage SecondaryData on Data objects

• API - Added support for "mounts" parameter when launching a pipeline to provide input mount paths for file and folder inputs

• API - Added fields to API error response schema: HTTP Method, User ID, Tenant Name, Tenant ID

• API – Added new endpoints for retrieving Entitled Bundles and to manage Project Bundles

Fixed Issues

• Docker images available via entitled bundles are listed in green text in the Docker Repository view

• Fixed an issue in OpenAPI spec causing invalid cURL requests in 'Try it Out' functionality for the create upload URL API

• Fixed an issue that resulted in empty Workgroup listing via the API and when adding Workgroups to a Project Team

• Fixed an intermittent error when navigating to Bundles view

• Fixed an issue where Tools available via Bundles are not displayed in the Tool Repository

• Fixed an issue where example JSON generation for root Metadata Model fails when the root Metadata Model is modified

• Fixed an issue where incorrect reporting of compute start times resulted in incorrect compute billing

• Fixed an issue where Nextflow analyses get stuck "In Progress" when output folder is deleted during an ongoing pipeline run

• Fixed an issue where incorrect casing on the scatter method 'dotproduct' in CWL auto-generated from graphical pipelines results in failures in pipelines with scatter-gather steps

• CLI - Fixed an issue with startAnalysis APIs no longer accepting the 'Accept' header with value "application/problem+json,application/vnd.illumina.v3+json”

• API/CLI - Fixed issue where graphical CWL pipelines launched via API/CLI do not use machine profiles (compute types) specified in the pipeline definition

• Fixed an issue where pipelines within shared bundles could not execute across regions

• Fixed an issue where exported Base Tables with null values populate a CSV with 'N' values instead of empty values

• Fixed an error when loading data into Base Tables with an invalid number of columns

• Fixed an issue causing View Results in Base Query History to return an expiration error

• Fixed an issue that caused JSON import into Base Tables to fail when sub-field (record type) is used in the Table schema

• Updated the error message when invalid SQL is used in a Base Query

• Fixed an issue with JSON import to Base Tables failing when reference data fields are enabled

• Fixed an issue where updated Base Table fields in the UI do not update the underlying Snowflake Table

• Fixed an issue resulting in "Unable to create file error" message when creating a Tool in a Bench Workspace

• Fixed an issue where Bench Workspaces cannot be deleted

• Fixed an intermittent issue where Bench Workspaces unexpectedly fail in an error state

• Updated messaging for users without access to Variant Viewer and Cohorts

• API - Fixed an issue where the API did not accept multiple 'Accept' header values

• API - Fixed an issue where the create folder upload sessions API returns date and time with incorrect formatting

• API - Fixed the 500 response on APIs when the user has not yet accessed the UI

• API - Fixed missing version header in OpenAPI spec for APIs to link a pipeline to a Project or Bundle

• API - Fixed the 500 response from create project API when the 'storageConfigurationSubfolder' value is invalid

• API - Fixed a 500 response from the API to link data to a bundle when linking a Folder with child data records

• API - Fixed an issue where the API to create File data record creates the data record at the root of the project data when a trailing forward slash ('/') is missing after the folder name

• API - Fixed an issue where API to retrieve pipeline reference sets does not display linked reference data

• API - Fixed an issue where APIs to initiate analysis perform validation of required properties in request body

2022 March - ICA v2.3.1

Fixed Issues

• Fix issue where pipeline launch is blocked when a single file is used as the input to multiple pipeline inputs

2022 March - ICA v2.3.0

Features and Enhancements

• A Base query is now possible with null marker 'nan'

• A user can search on Secondary Data tab

• A user is allowed to run a pipeline from a shared project if the Billing mode is set as 'Tenant'

• Add "application" field to Project API model to indicate whether a project is externally managed.

• An error is shown when a user tries to build a workspace that is not running.

• An error is thrown when the user uses the same name of an existing project connector rule in the same project

• Base query can be exported as a view

• Bug fix for allowing multiple files with same name but different paths to be passed as input; causes conflict when downloading inputs to pipeline filesystem, one file will be overwritten (random) causing unpredictable results

• Data filters and columns now also apply to technical tags

• Externally managed projects are now supported via API

• Meta data schedule can be created on tenant level

• New ICA notification for Data/Sample linked

• Preview of the content of a Base table is available with null marker nan

• Project connector permissions are aligned with manual linking permissions

• Remove obsolete "BlueVantage" option from the Source drop down selections during Base schedule creation

• Revert cwltool upgrade

• Samples linked via a project connector are visible in the 'receiving' project

• The 'Type of Files' field in the configuration of Upload Rules in a Connector is changed to 'Assign Format' so that its purpose is clear

• The project connector rules are available across tenants when the user has Contributor rights

• The user is able to use the filters 'Type' and 'Status' and sort the files

• The yaks-scheduler used to check the pod-to-node fit, but will now defer to the kubernetes default-scheduler as the pod-placement authority. The assumption is that when the pod is not scheduled, there are no nodes available, and a node will be provisioned.

• When a user links a sample from another project to the current project, the data will not be automatically linked to the current project as well. The user should explicitly add the data to the current project as well if the user wants to use it.

• [Base] When selecting 'Include reference', you will see the record first and when you unfold the record, you will see the fields belonging to the record

• When trying to populate the Sample Metadata, we now always use the Project Metadata Model if there is no Pipeline Metadata Model.

• [API] Add ability to filter by region for list projects API

• [API] Add psToken authorization type to enable creating JWT from psToken via the create token API

• [Bench] Add ICA_PROJECT_UUID environment variable to the Jupyter Notebook Bench image.

Fixed Issues

• Fix 400 error response when launching an analysis with a pipeline from a bundle

• Fix Base Query results file export set to format UNKNOWN

• Fix Bench pricing calculation accuracy

• Fix for Base table export unexpected failure

• Fix for auto-linking pipeline outputs files to a sample when input files provided are linked to the sample

• Fix for failed Base table import when imported data contains "nan" (null marker) for a column of type integer

• Fix for linking Base Query templates to Bundles

• Fix for modifying Bench Workspace image unless Workspace is in Stopped state

• Fix for more informative API error response when EULA has not been accepted

• Fix for preserving sample data links for samples linked from shared bundles

• Fix for retrieving download URL for data in an externally managed project.

• Fix for silent error when running Base schedule on Administrative Data or Metadata

• Fix intermittent UI issue where checkbox for file selection during docker image add is partially visible

• Fix issue where Base schedule for Metadata does not work when source is set to "Tenant"

• Fix issue where Base table schema definition changes from record to string after sync

• Fix issue where Project Connectors cannot link samples where the source project is owned by a different tenant

• Fix issue where an analysis cannot be started when a metadata model is defined for the pipeline

• Fix issue where data cannot be removed from a Sample using the "Remove Data" button from the sample details view.

• Fix issue where pipelines from linked bundles do not show as available selections in the "Generated by Pipelines" drop down when creating a Base schedule

• Fix issue where samples cannot be updated after configuring a Project Connector rule with the sample.

• Fix issue where scroll bar is not shown in the metadata models list view

• Fix issue where secondary data is not shown in the reference data view

• Fix issue where team members added to eternally managed projects cannot be granted Contributor permissions, and cannot link data to other projects

• Fix issue where user cannot use sample filtering when samples and files are manually linked to their project and the files do not appear associated with their sample in that project

• Fix issue where user is unable to see their input files when using sample filtering if they are linked from another project without selecting the flat view option

• Fix issue where workgroups cannot be added to project team of externally managed projects with Contributor permission

• Handle K8s api timeouts by retrying query.

• Improve pipeline analysis startup times to approximately 10 mins maximum (times may vary)

• [API] Fix 500 response for getProjectDataLinkedProjects API

• [API] Fix error response when invalid API Key is provided for authorization

• [API] Fix get bundles API error response when user has access to a project with a linked bundle that the user does not have access to

• [API] Fix getProjectData API to return correct results when using the "id" query param

• [API] Fix incorrect error response for Project Data API to create folder upload session

2022 February - ICA v2.2.0

Features and Enhancements

• the cwl workflow engine logs now also show debug logs, these includes the javascript expressions evaluated etc. these logs are similar to running cwltool --debug locally

• Show "username" of pipeline creator in Pipelines overview view

• Removed excessive logging from the "Audit Trail" logging.

• Various performance improvements - reuse of connections to Base instances

• A particular storage configuration should only be available during project creation in case it is your own storage configuration, or when the owner (or a tenant administrator) gives explicit access to it for other users in their tenant.

• Bench - when trying to get an access token through the API, access was denied in case Base was not enabled within the project.

• Base - 'You are not allowed to modify data' was shown to the user when clicking the Sync button as a Contributor

• Base - importing data with format JSON in a table was not possible.

• Files within a sample will only show up in case the data is also explicitly linked to the project/bundle.

• Bundles - a user with Contributor access did not have the necessary access rights for managing all resource of the bundle.

• Project billing modes can be 'project' or 'tenant'. In case 'tenant', the users from another tenant than the owner of the project was not able to upload files.

• Within the Activation Codes view, activation codes for access to the Variants/Cohorts modules were not listed.

• It was not possible to add a bundle at project creation. The project had to be saved before the bundle could be added.

• The roles explanation in the Team page on a permission record did not show the entire information.

• Various stability improvements - In case of a database outage, ICA was not able to automatically reconnect.

• When managing docker images, a partial docker image was not allowed to be deleted nor updated resulting in a stuck image.

• When exporting the contents of a grid from the UI, the exported file contained tenant ids instead of tenant names.

• Base - the export of query results to a json format resulted in a failure in case the query ended with ';'

• A non-tenant-admin user was not able to create a metadata model.

• Base - exporting query results, specifying a custom delimiter did not work.

• When a user clicks the bundle card in the "Bundles" view, and the user has no access to manage the bundle because it is shared with the user by another tenant, the following notification will be displayed: "Details for a bundle from another tenant are not accessible" instead of "Bundle details unavailable"

• When a user links a sample from another project to the current project, the data will not be automatically linked to the current project as well. The user should explicitly add the data to the current project as well if the user wants to use it.

• Various performance improvements - linking a folder with a lot files to another project or bundle timed out.

• Base - loading metadata into a table using a schedule resulted in the same data being added over and over again.

• When trying to populate the Sample Metadata, we now always use the Project Metadata Model if there is no Pipeline Metadata Model.

• Limit usable character set for the user reference of an analysis run

• When an analysis was aborted, the output files which were generated so far were not visible in the details of the analysis, but only in the data view.

• Base - schedule metadata population in a table did not work.

• Added support for Docker Images to be added to Bundles

• Added Project Connectors in the project connectivity section to enable creation of automatic sample and data linking rules between projects.

• Added Queue Date to analysis Logs view to indicate when the specified job was submitted to the underlying infrastructure.

• Added support for externally-managed projects for Illumina software applications built on top of ICA to manage resources inside ICA projects while limiting tenant users' direct access to the project to read-only capacity.

• Improved performance of API calls when using API Key for authentication

• Added tenantName to API resource DTOs

• Add API to abort analysis run

• Added APIs to share a storageConfiguration, share a storageCredential, update a storageCredential secret

• Added mime-type field to the DataFormat DTO for the data APIs

• Added "objectEtag" field to DataDto for Data APIs to expose e-tag from the cloud provider object store

• Added Base APIs to retrieve Jobs, retrieve Tables, and load data into Table

• API - POST‚/api/projects/{projectId}/samples:search was missing request parameters to browse through samples in a paged manner.

• API - POST /api/projects/{projectId}/analysis:cwl - tags were not persisted when starting a CWL analysis run

• The PUT /project/{projectid}/data/{dataid} gives the user the impression that all fields can be updated on the data, but it is a different set for files as for folders. This distinction is not properly documented.

• API - GET /api/projects/{projectId}/analyses - when the pipeline did not belong to the tenant, but the user still has access to it (eg: through a bundle), the analysis details could not be retrieved through the API.

• API - GET /api/projects/{projectId}/analyses/{analysisId} returned the details of an analysis, even if the projectid did not match the actual project the analysis ran in.

• API - GET /projects/{projectId}/samples/{sampleId} did not expose metadata information

• API - GET /projects/{projectsId}/data presented the user with an unclear error message in case providing invalid request parameters.

• API - An unclear error was presented in the problem response in case an invalid json was provided as request body.

• API - An unlcear error was presented in the problem response when creating a project using invalid uuids in the request body.

• API - Removed deprecated "sample matching strategy" endpoint

• CLI - Added command to update project data

• CLI - Added command to retrieve a list of project samples

• CLI - Added command to create a project sample

• CLI - Added command to list available data formats

• CLI - API Key encryption modified to use local value on machine where API Key is configured to prevent hijacking config.

• CLI - Added command to get details of a project sample

• CLI - Added command to list data linked to a project sample

• CLI - Added command to update a project analysis

• CLI - Added command to update a project sample

• CLI - Added command to unlink data from a sample

• CLI - Added commands to delete a project sample

• CLI - Added command to link data to a sample

• CLI - Added command to complete a sample

• CLI - Removed unnecessary flags on the config command.

• CLI - For starting CWL and Nextflow pipelines, the flag 'storage-size' was ignored by CLI, and the default storage size of the pipeline was taken.

Fixed Issues

• [bugfix] when CWL workflows failed or produced null as one of the outputs, all the outputs were incorrectly processed. in this case, the user may not be able to see the details of the analysis output in UI/API.

• [Base] Fix error message when viewing a query from query history that no longer returns results

• [bugfix] in some cases after aborting an analysis, it may remain stuck in aborting state indefinitely. Now more resources are explicitly cleaned to avoid this issue from happening.

• Fixed bug when showing logfiles of steps in an analysis.

• [bugfix] cwl workflow sometimes fails in spite of all steps in the workflows have succeeded. Made the workflow engine robust to detect and retry in this case

• When linking a folder containing +2000 files to another project, only 2000 files were actually linked.

• When a user did not choose a country when filling out the user account details, login to ICA could fail.

• Fixed bug when handling custom output objects in CWL.

• Fixes issue with compute type selection when launching CWL graphical pipelines

• The Project > Team permissions showed a little pencil in front of the row indicating a modification. With this fix, the pencil will only show when there were actual modifications done to the row in particular.

• API - All dates in API responses were formatted in 12 hours notation without AM/PM notation. This way a time in PM is presented as AM. With this fix, all time notations are in 24 hours.

• When starting an analysis through the sample page, the start window is not closed after pressing start. Users would click it again, which would start the analysis multiple times. With this fix, the start window is closed after pressing start.

• When starting an analysis fails because the backend service returns an error, the status of the analysis is "Awaiting Input". With this fix, the analysis goes to "Failed".

• Base - in case the export of query results failed, an empty file was created and never cleaned up. With this fix, there are no leftovers in case of export failure.

• When a sample was linked from another project to the current project, the details view of the sample shows all data linked to it, even of the current project does not have access to it. With this fix, only the data which is also linked to the current project will be shown on the details of the sample.

• API - PUT /project/{projectid}/data/{dataid} update sometimes had to be done twice for it to persist. With this fix, all editable attributes can be updated at once.