Data Transfer

The ICA CLI is a useful tool for uploading, downloading and viewing information about data stored within ICA projects. If not already authenticated, please see the Authentication section of the CLI help pages. Once the CLI has been authenticated with your account, use the command below to list all projects:

icav2 projects list

The first column of the output (table format, which is default) will show the ID. This is the project ID and will be used in the examples below.

Upload Data

To upload a file called Sample-1_S1_L001_R1_001.fastq.gz to the project, copy the project id and use the command syntax below:

icav2 projectdata upload Sample-1_S1_L001_R1_001.fastq.gz --project-id <project-id>

To verify the file has uploaded, run the following to get a list of all files stored within the specified project:

icav2 projectdata list --project-id <project-id>

This will show a file ID starting with fil. which can then be used to get more information about the file and its attributes:

icav2 projectdata get <file-id> --project-id <project-id>

It is necessary to use --project-id in the above example if not entered into a specific project context. In order to enter a project context use the command below.

icav2 projects enter <project-name or project-id>

This will infer the project id, so that it does not need to be entered into each command.

Note: filenames beginning with / are not allowed, so be careful when entering full path names as those will result in the file being stored on S3 but not being visible in ICA. Likewise, folders containing a / in their individual folder name and folders named '.' are not supported

Download Data

The ICA CLI can also be used to download files via command line. This can be especially helpful if the download destination is a remote server or HPC cluster that you are logged into from a local machine. To download into the current directory, run the following from the command line terminal:

icav2 projectdata download <file-id> ./

The above assumes you have entered into a project context. If this is not the case, either enter the project that contains the desired data, or be sure to supply the --project-id option in the command.

Temporary Credentials

To fetch temporary AWS credentials for given project data, use the command icav2 projectdata temporarycredentials [path or data Id] [flags]. If the path is provided, the project id from the flag --project-id is used. If the --project-id flag is not present, then the project id is taken from the context. The returned AWS credentials for file or folder upload expire after 36 hours.

Data Transfer Options

For information on options such as using the ICA API and AWS CLI to transfer data, visit the Data Transfer Options tutorial.