The library file management tool in Partek Flow provides an easy way to create, process and manage reference sequences, cytoband files, annotation models, aligner indexes, gene sets, variant databases and microarray probe sequence files.

For a quick start, please watch the short videos below:

Quickly setup library files for the hg19 human reference genome assembly.

The process can be adapted to the different kinds of file types.

Adding library files for a non-model organism

You can watch the full webinar on our website.

This user guide will cover the following topics:

• Library File Management Settings

• Library File Management Page

• Selecting an Assembly

• Library Files

• Update Library Index

• Creating an Assembly on the Library File Management Page

• Adding Library Files on the Library File Management Page

• Adding a Reference Sequence

• Adding a Cytoband

• Adding Reference Aligner Indexes

• Adding a Gene Set

• Adding a Variant Annotation Database

• Adding a SnpEff Variant Database

• Adding a Variant Effect Predictor (VEP) Database

• Adding an Annotation Model

• Adding Aligner Indexes Based on an Annotation Model

• Adding Library Files from Within a Project

• Microarray Library Files

• Adding Prep kit

• Removing Library Files

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

On the library file management page, an assembly can be added by choosing Add assembly… from the Assembly drop-down list (Figure 1). If the list is long, you may need to scroll to the bottom.

In the Add assembly dialog, choose the Species from the drop-down list (Figure 2), followed by the Assembly version (Figure 3) and click Add. The dialog will automatically load commonly used assembly versions for the selected species (Figure 3). If the assembly version you want does not appear in the list, choose Other and type the custom assembly version name (Figure 4). Characters such as $ * | \ : " < > ? / % cannot be used in custom names. Note that If an assembly version for a given species already exists on your system, it will not appear in the drop down list.

If the species you want to add does not appear in the list, scroll to the bottom of the species list, choose Other and manually type the species name and assembly version (Figure 5).

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

The library files associated with the selected assembly are organized into . Below is some information on each section.

Reference Files

This section includes two types of library file: reference sequence and cytoband files.

Reference sequences are the chromosome/scaffold/contig DNA sequences for a species. A reference sequence file is typically in FASTA or 2bit format. The reference sequence of a species is used for aligner index creation, variant detection and visualization of the reference sequence in the Chromosome view.

Cytoband files are used for drawing ideograms of chromosomes in the , including positions of cytogenetic bands if known.

Reference aligner indexes

Next-generation sequencing aligners require the reference sequence to be indexed prior to alignment, as this greatly increases alignment speed. An index consists of a set of files (Figure 1) and are generally aligner specific. For example, if you wish to align using BWA, you need a BWA index.

Some of the supported aligners share indexes. If you want to align using Tophat, the Bowtie aligner indexes can be used. If you want to align using Tophat2, the Bowtie2 aligner indexes can be used.

Some aligner indexes are version specific, so care must be taken if you change aligner versions. For example, the index files for STAR version 2.4.1d are different to older versions of STAR.

This section contains aligner indexes for aligning to the whole genome. If you wish to align to a subset of the genome, e.g. targeted amplicons or the transcriptome, you must generate these indexes in the Annotation models section.

Gene sets

Variant annotations

Variant annotation databases are collections of known genomic variants (e.g. single nucleotide polymorphisms). If you have performed a variant detection study, detected variants can be searched against variant annotation library files to see if the detected variants are known from previous studies. Furthermore, you can validate detected variants against 'gold-standard' variant annotation library files. Variant annotation files are typically in VCF format.

SnpEff variant databases

VEP database

Annotation models

This section includes two types of library file: annotation models & aligner indexes.

Annotation models describe genomic features (e.g. genes, transcripts, microRNAs) for a specific version of the reference sequence. Annotation models contain labels (e.g. gene ID) and genomic coordinates (e.g. chromosome, start & stop position) for each feature.

Annotation models will appear in separate tables (Figure 2). If you have multiple versions of annotation models from the same source, it is advisable to distinguish them by their date or version number.

References

1. Cingolani P. et al. A program for annotating and predicting the effects of single nucleotide polymorphisms, SnpEff: SNPs in the genome of Drosophila melanogaster strain w1118; iso-2; iso-3. Fly. 6(2):80-92. PMID: 2272867

Additional Assistance

Choosing an Assembly from the drop-down list at the top will display the associated library files in the panel below. Assemblies are named by the species and the build version (e.g. Homo sapiens (human) - hg19). Different build versions for the same species are regarded as separate assemblies (Figure 1). Administrative users can delete a selected assembly by clicking the bin () icon.

Additional Assistance

Click the + Add reference files button under the Reference Files section header and choose Reference sequence from the Library type drop-down list in the Add reference sequence dialog (Figure 1). If a cytoband file is already associated with an assembly, Reference sequence will be the only option available in the Add reference sequence dialog and will not appear in a drop-down list (Figure 1).

If you are using an assembly supported by Partek (e.g. human), select Download reference sequence and click Create to get the reference sequence from the Partek repository. Alternatively, select Import reference sequence and click Create to add the reference sequence from another source.

If you are using a custom assembly (e.g. for a non-model organism), the Download reference sequence option will not be available (Figure 2).

The Import reference sequence option allows you to add a reference sequence from the Partek Flow Server or a URL download link (Figure 3).

To add a reference sequence from the Partek Flow server, select Partek Flow Server, click Browse icon and navigate to the directory storing the reference sequence file(s) (Figure 4). Only files with the correct file extensions (.fa, .2bit or any compressed format) will be displayed in the file browser. Select the checkbox next to the file name(s) you wish to import and click Continue (Figure 4). Click Finish to add the selected reference sequence file(s).

To upload a reference sequence from your local computer, click Transfer files to the server, click Transfer files, drop files to the Transfer files dialog or click My Device and navigate to the location of the file(s) on your local machine (Figure 5). Click Upload to complete the file upload. Do not exit the browser tab or let the computer go to sleep or shut down until the transfer has completed.

To download a reference sequence (e.g. from a public repository such as ENSEMBL), select URL and input the URL address (Figure 6). Make sure to include the full URL address, starting with the transfer protocol (e.g. http:// or ftp://) and ending with the file name. It is advisable to find the download link address, copy it to the clipboard and paste it into the Input URL box. Click Finish to add the selected reference sequence file(s) (Figure 6).

Note that you can specify multiple fasta files (e.g. one file per chromosome) and Partek Flow will concatenate them. If a fasta file is imported, a 2bit version of the file will automatically be created and vice-versa. Partek Flow will also accept a range of compressed file formats (gzip, bzip2, zip, tar). If a compressed file is imported, it will automatically be uncompressed and processed.

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

Click the + Add gene set databases button under the Gene set databases section header on the library file management page.

If you are using an assembly supported by Partek (e.g. human), a gene set from geneontology.org will appear in the Gene set database drop-down list (Figure 1). Select the Download gene set database radio button and click Create.

If you prefer to add a custom gene set, or if you are working with a custom assembly, choose Add gene set database from the Gene set database drop-down list (Figure 2). Name the gene set by typing into the Custom Name box and click Create. Characters such as $ * | \ : " < > ? / % cannot be used in custom names. A gene set file can be added from the Partek Flow Server or a URL download link. The behavior of each option is similar to when importing a reference sequence (see Adding a Reference Sequence). When browsing for files on the Partek Flow server, only the files with relevant file extensions will be visible (.gmt and various compressed formats).

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

To access the library file management page click the avatar in the top right corner and choose Settings. Then under Components on the left, click Library files.

The library files page has three tabs - Genomic library files, Prep kits files, and Other library files. This section of the user guide will focus on the Genomic library files tab, which is relevant for next-generation sequencing analysis (Figure 1).

The arrows ( v / ) expand/collapse each section. Associated library files are shown in a table in each section. In the Actions column of each table, the View library file details button displays additional library file details, the bin () icon dissociates a library file. The hourglass () icon indicates a library file is being created.

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

Click the + Add variant annotations button under the Variant annotations section header on the .

If you are using a human - hg19 assembly, variant annotation databases from various sources will appear in the Variant annotation drop-down list (Figure 1). Available variant annotation database sources include:

• dbSNP

• Kaviar

• NHLBI Variant Server

• 1000 Genomes

Multiple versions of the above databases are available. For human - hg38, only dbSNP is currently available. This list is periodically updated.

Choose a database from the drop-down list, select the Download variant database radio button and click Create.

Additional Assistance

Library files can be added to an assembly by clicking the + Add reference files button under the Reference files section (Figure 1) on the library file management page. If all possible library files are already associated in the section, the Add reference files button will be disabled (Figure 1).

Clicking + Add reference files button opens a dialog specific to the section. Choose a library type from the drop-down list in the Add reference sequence dialog (Figure 2) and the dialog will present options specific for each type. If certain library files have already been associated with an assembly, they will not appear as options in the dialog. For example, if a reference sequence has already been associated with an assembly, it will not appear as an option in the drop-down list shown in Figure 2.

Additional Assistance

Note that this task is for adding indexes for alignment to the whole genome. If you want to align to the transcriptome or another set of genomic features, see .

Click the + Add reference aligner indexes button under the Reference aligner indexes section header on the . Choose the aligner index you wish to add from the drop-down list in the Add aligner index dialog (Figure 1). The following indexes can be added:

• Bowtie

• Bowtie 2

• HISAT 2

• TMAP

• BWA

• GSNAP

• GSNAP v8

• STAR

• STAR 2.5.3a

• STAR 2.7.3a

• STAR 2.7.8a

• Issac 2

• RSEM

• Salmon

• Cell Ranger ARC 2.0.0 reference

• Minimap2

If you are using an assembly supported by Partek (e.g. human), there are three radio button options: Download index; Build index or Import index (Figure 1). Certain aligner indexes may not be available for automatic download because the file sizes are too large to download efficiently.

If available, select Download index and click Create to get the chosen reference aligner index from the Partek repository.

Alternatively, select Build index and click Create to build the reference aligner index. To build an aligner index, a reference sequence file must already be associated with the assembly. Depending on the aligner, you may have to specify further parameters. Consult the user documentation for each aligner for guidance (usually available on-line).

For custom assemblies (e.g. for non-model organisms), only the Build index and Import index options are available (Figure 2).

Additional Assistance

The library index is a file that contains download sources for all Partek distributed library files. An automatic update of this file will occur every 24 hours (Figure 1), so this normally doesn't require any attention. If you do wish to manually update the library index, click Update Library Index at the bottom of the .

Additional Assistance

Click the + Add SnpEFF variant databases button under the SnpEff variant databases section header on the library file management page.

If you are using human (hg19 and hg38), mouse (mm10) or rat (rn5 and rn6) assemblies, various versions of SnpEff variant databases are available for automatic download (Figure 1).

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

On the library file management page, click the + Add annotation models button in the Annotation models section, choose Gene/feature annotation from the Library type drop-down list in the dialog (Figure 1).

If you are using an assembly supported by Partek (e.g. human), annotation models from a variety of commonly used sources (e.g. ENSEMBL, GENCODE) will appear in the Annotation model drop-down list in the dialog. Choose an annotation model, select the Download annotation file radio button and click Create (Figure 1).

If you prefer to add a custom annotation file or if you are working with a custom assembly, choose Add annotation model from the Annotation model drop-down list (Figure 2). You may need to scroll to the bottom of the drop-down list to see this option. Name the annotation model by typing into the Custom name box and click Create. Characters such as $ * | \ : " < > ? / % cannot be used in custom names.

A custom annotation model can be added from the Partek Flow Server or a URL download link. The behavior of each option is similar to when importing a reference sequence (see Adding a Reference Sequence). When browsing for files on the Partek Flow server, only the files with relevant file extensions will be visible (.gtf, .gff, .gff3, .bed, .pannot, .txt and various compressed formats).

Select the source of annotation file and click Next. If the annotation file format is gtf, gff, gff3 or bed, a preview of the first 10 rows of the file will be shown on the screen (Figure 3). You must then specify the type of annotation file by choosing an option from the Annotation type drop-down list (Figure 3). Choose mRNA for whole transcriptome annotations, where both gene and transcript level information are present. Choose microRNA for precursor or mature microRNA transcript annotations. Choose Amplicon for targeted amplicon sequencing annotations. Choose Other for annotation files that do not fall into the other four categories (e.g. lncRNA).

For text annotations with both gene- and transcript-level information select mRNA from the drop-down menu. You can manually specify which column corresponds to the Transcript ID or Gene ID by selecting the corresponding radio button next to the column with that information (Figure 4).

Genome coordinates for annotation models stored in Partek Flow are 1-based, start-inclusive, and stop-exclusive. This means that the first base position starts from one, the start coordinate for a feature is included in the feature and the stop/end coordinate is not included in the feature. These are the genome coordinates that are printed in various task reports and output files when an annotation model is involved in the task. When custom annotation files are added to Partek Flow, the genome coordinates are converted into this format. The coordinates are converted back if necessary for a specific task. Figure 5 shows how the genome coordinates vary between different annotation formats.

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

To review and edit library file management settings, you must be logged into Partek Flow as a user with administrator privileges. Click on the avatar in the top right corner and choose Settings. Then click System preferences on the left.

You can review the current library file directory location (Figure 1).

To change where the library files are stored, click Edit filesystem and storage and click Browse () icon to point to another directory (Figure 2).

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

Click the + Add VEP database button under the VEP database section on the library file management page.

If you are using an species supported by Partek (i.e., human, mouse, rat), a VEP database from ENSEMBL will be available for automatic download. Select the Download database radio button and click Create (Figure 1).

If you are working with a custom assembly, choose the Import database radio button and click Create (Figure 2). A VEP database can be added from the Partek Flow Server or a URL download link. The behavior of each option is similar to when importing a reference sequence (see Adding a Reference Sequence). ENSEMBL maintains an extensive list of VEP databases for various species. We recommend going to ENSEMBL's ftp download site and finding the VEP database download link for your species of interest.

One VEP database is allowed per genome assembly. Once a VEP is successfully imported, the + Add VEP database button will not be shown (Figure 3).

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

Missing library files can be added when setting up tasks within a project, without having to navigate to the library file management page. The user interface will vary depending on the task and which library files already exist on your system. Below are two examples scenarios.

Adding a missing whole genome aligner index (Bowtie 2 in this example)

1. Under the Analyses tab of a project, select an Unaligned reads data node

2. From the context sensitive menu on the right, choose Aligners followed by Bowtie 2. On the alignment task setup page, Partek Flow will display all assemblies that have a Bowtie 2 index (whole genome and transcriptome) in the Assembly drop-down list. If the assembly you want is missing, choose New assembly… from the drop-down list (Figure 1)

1. Choose the species and assembly in the Add Bowtie 2 index dialog. If the species and assembly you want do not appear in the drop-down lists, choose New species... and manually type the names (Figure 2)

2. Choose Whole genome from the Index drop-down list (Figure 2)

3. Select the Build index radio button (Figure 2)

4. Click Create (Figure 2)

Once the new Bowtie 2 index has been specified, you are able to queue the alignment task and it will execute once the Bowtie 2 index has been built.

Add a missing gene set file for enrichment analysis

1. Under the Analyses tab of a project, select a Feature list data node

2. Choose Biological interpretation from the menu on the right, followed by Gene set enrichment

3. Select Gene set database radio button for Database. If the species and assembly you want do not appear in the Assembly drop-down list, select New assembly... (Figure 3)

1. If you are working with an assembly/species supported by Partek (e.g. human), choose a gene set from the Gene set database drop-down list (Figure 4), select the Download gene set database radio button and select Create. Alternatively, choose Add gene set database from the Gene set database drop-down list, manually type the custom gene set name and click Create to import your own gene set from the Partek Flow server or URL (Figure 5). Characters such as $ * | \ : " < > ? / % cannot be used in custom names. If you are working with a custom species/assembly (e.g. for a non-model organism), only the Add gene ontology source option is available.

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

Click the + Add reference files button under the Reference Files section header and choose Cytoband from the Library type drop-down list in the Add reference sequence dialog (Figure 1). If a reference sequence is already associated with an assembly, Cytoband will be the only option available in the Add reference sequence dialog and will not appear in a drop-down list (Figure 1).

If you are using an assembly supported by Partek (e.g. human), select Download cytoband and click Create to get the cytoband file from the Partek repository. Alternatively, select Build cytoband and select .fasta files or .2bit files to to build the cytoband file. If the reference sequence is missing, it will either be downloaded automatically or you will be asked to import it from another source (see Adding a reference sequence).

If you are using a custom assembly (e.g. for a non-model organism), the Download cytoband option will not be available (Figure 2).

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

This document will show you how to delete an assembly and how to dissociate a library file.

Deleting an Assembly

To delete an assembly from the Library File Management, select the Delete assembly () icon by the assembly name. You will be prompted to confirm the deletion (Figure 1). If you also want to delete the individual library files from the disk, select the Delete the library files check box. Select Delete to proceed with deletion. Note that the deletion can not be un-done.

Disassociate Library Files

Individual library files can be removed by disassociating them from their respective assembly. To disassociate a file, click on Disassociate library file in the Actions column (Figure 2).

If the library file is used by multiple projects, the resulting dialog will provide a list of projects (Figure 3). As long as the library file is used, the Delete the library files option will be disabled.

Using the Dissociate library file tool does not physically remove the file from the disk, but disassociates the file from the selected library file name. This, in turn, enables you to associate newer or improved version of the library file with the same library file name. If the goal is to physically delete the library files from the disk, then all the projects using that library file must be deleted first.

Microarray library files can be managed under Microarray library files section on Other library files tab on the Library file management page (Figure 1). The chip name and download source of stored Microarray library files will be shown in the table in this section. For more information, refer to the section.

Microarray probe tab files are used for processing microarray data in Partek Flow. When microarray intensity data files (e.g. Affymetrix .CEL files) are imported into a project, the chip type is automatically detected and the appropriate probe tab annotation file is downloaded. Thus, you would normally not need to manually add any probe tab annotation files.

To add a (e.g. for a custom chip), click the + Add probe sequence button at the Microarray library files section (Figure 1). Scroll to the bottom of the Chip name drop-down list and choose Other / Custom. Name the chip by typing into the Custom name box and click the Create button (Figure 2). Characters such as $ * | \ : " < > ? / % cannot be used in custom names.

Additional Assistance

Note that this task is for adding indexes for alignment to a subset of the genome (e.g. the transcriptome). If you want to align to the whole genome, see Adding Reference Aligner Indexes.

On the library file management page, click the + Add annotation models button at the Annotation models section and choose Aligner index from the Library type drop-down list in the dialog (Figure 1).

Choose the aligner you wish to use from the Aligner drop-down list (Figure 1). All aligners are available for indexing to an annotation model.

The annotation model(s) that have already been associated with an assembly will appear at the top of the Index to drop-down list. Choose the annotation model you wish to index to, select the Build index radio button and click Create (Figure 1). To build an aligner index based on an annotation model, a reference sequence file must already be associated with the assembly.

If you are using an assembly supported by Partek (e.g. human), annotation models from a variety of commonly used sources will appear in the Index to drop-down list in addition to the ones that have already been associated with the assembly. If you choose an annotation model that has not already been associated, it will automatically be downloaded prior to building the index.

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.

Partek distributes prep kit files for a variety of single cell technologies, such as 10x Genomics, Drop-seq, and Fluidigm C1. Prep kit files are required to process single cell fastq files. If you need to add a new prep kit in order to process your fastq files, you will need to get detailed information on how the library is constructed for the specific assay.

The following instructions use the 10X Genomics Chromium single cell gene expression 3' v3 chemistry as an example to illustrate how a prep kit tag library file is made in Partek Flow. Figure 1 is a schematic diagram of the single cell 3' v3.1 gene expression assay:

Click + Add prep kit button on Prep kit files tab at Library files management page, choose Other/Custom option from the Prep kit name drop-down list and specify a name e.g. Assay1 (Figure 2), choose the Build Prep kit option and click Create.

Select the Is paired end check button, and specify the information contained in each end:

Read 1: contains 10X barcodes and UMI, it is 28bp long. 10X Genomics has a barcode whitelist which contains all known barcode sequences during library preparation. You will need to make a .fasta file format of this information and click on the + icon in the First mate segmentation (Figure 3) to add the barcode as the first segment in read 1.

Specify the new segment (Figure 4), choose Barcode from the Type drop-down list, specify the barcode .fasta file. If the barcodes are random with a fixed length, choose Arbitrary and specify the length. You can also choose the Manual option to manually add the barcode sequences that are available for this assay. Specify number of mismatches are allowed. Click Add

Click on the remove (-) icon to remove a segment if you make a mistake (Figure 5). Click on the + icon next to Barcode to add more segments, e.g. UMI.

According to the 10X 3' v3 assay, the UMI is a 12bp sequence (Figure 6).

Read 2 contains the 90bp insert sequence, click on the + icon next to Second mate segmentation, choose Insert from the Type drop-down list and specify the minimum read length allowed (Figure 7).

Check the Remove poly-A tail button and click Finish. The new prep kit file is added to the library file database in Partek Flow (Figure 8).

Click the three dots under Action, select View prep kit details to view the diagram (Figure 9).

When you perform the trim tag task on fastq files, Partek Flow will remove read 1 and the downstream alignment task will only be performed on read 2. The barcode and UMI information will be added in the read name in the output file for downstream analysis.

Additional Assistance

If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.