Latest improvements for QIAGEN CLC Genomics Workbench

Improvements

• When exporting images to bitmap-based formats, the Screen resolution and High resolution options are now bounded so the maximum supported number of pixels will not be exceeded.
• Search for Sequences in Uniprot has been updated to reflect changes at UniProt Knowledgebase (UniProtKB). This tool is broken in earlier releases, including CLC Genomics Workbench 22.0, 22.0.1 and 22.0.2.
• Third party plugins for CLC Workbenches can be installed when the Workbench is running in Viewing Mode.
• Various minor improvements

Bug fixes

• Fixed an issue causing Map Reads to Reference to fail if a masking track covering a whole chromosome was provided as input.
• Fixed an issue in RNA-Seq Analysis that could lead to an IndexOutOfBounds error when the option “Calculate expression for genes without transcripts” was selected, and two or more genes had the same name, and at least one of these has no transcripts, and the Region column of the table view of the gene track contains the text “join”, “>”, or “<” (i.e., the genes have splice structure, or uncertain end positions).
• Fixed an issue affecting Extract Consensus Sequence where annotations transferred from the reference sequence to the consensus sequence could be wrongly positioned if the read mapping had an insertion in a region that was removed due to low coverage.
• Amino Acid Changes
  • Fixed an issue causing the output from to be named after the reference data instead of the input data.
  • Fixed an issue that caused the transcripts and proteins listed in the Coding region change and Amino acid change columns in the annotated variant track output to be inconsistently ordered.
• Fixed issues with Quantify miRNA where:
  • It would fail on paired reads if using spike-ins.
  • Opening a sequence list to view it would cause this tool to fail if that same sequence list had been used as input.
• Fixed issues affecting the right-click option to ”Extract Sequence…” over a sequence track in a track list containing an annotation track:
  • The “Extract annotations” setting was having no effect. Even when unchecked, annotations were included in the output.
  • The “Extract annotations” option was disabled when no sequence region had been selected before running “Extract Sequence”. In this situation, the option is now enabled, and turned on by default.
• Fixed an issue setting the subtree line color for a particular node of a phylogenetic tree would result in that color also being applied to neighboring nodes.
• Fixed an issue that could cause the Workbench to freeze when exporting elements with certain view settings to graphics formats, for example read mappings with compactness set to “Not compact” and the Sequence layout set to “No wrap”.
• Fixed an issue causing Standard Import of GenBank format to stall if qualifier names spanned more than one line.
• Fixed an issue where unlocked options for on-the-fly importers in a workflow would be locked if the Input element was re-opened for editing.
• Fixed an issue where upgrading on Windows systems could be blocked due to a locked file.
• Fixed an issue where text in installer screens was not visible when installing the software in ‘dark mode’ on Linux.

Changes

• The Java version bundled with CLC Genomics Workbench 22.0.3 is 11.0.17, where we use the JRE from the Azul Zulu Builds of OpenJDK.
• PFAM accessions in the results table created by PFAM Domain Search are linked to PFAM entries hosted by InterPro. Links generated in older versions of the software were to the Pfam website, which is being decommissioned.

Bug fixes

• Fixed an issue affecting Local Realignment where false positive insertions and deletions could be introduced in read mappings when a guidance variant was provided that had a sequence similar to the reference sequence immediately after the variant. See Important QIAGEN CLC software notifications for further details.
• Fixed an issue affecting the naming of outputs when Local Realignment was provided with multiple read mappings as input. When the tool was not run in Batch mode, a read mapping was generated per input, as expected, but all the realigned read mappings were named after the first read mapping in the set of inputs. Now each is named after the corresponding input.
• Fixed an issue with Predict Secondary Structure (RNA), where the Secondary Structure 2D View did not show colors when coloring by “Base pair probability” or “Unpaired probability”.
• Fixed an issue where Template Workflows delivered by a plugin could be listed twice in the Toolbox after an unsuccessful plugin installation or removal.
• Fixed an issue that could result in an error like “no zstd-jni in java.library.path” when importing files or undertaking other activities involving the compression of CLC data elements.

Plugin notes

• After upgrading the CLC Genomics Workbench:
  • Immune Repertoire Analysis will be able to identify clonotypes from data with a varied read structure. This tool is available when the Biomedical Genomics Analysis plugin is installed.
  • Single Cell TCR-Seq Analysis will be able to identify clonotypes from data with a varied read structure. This tool is available when the CLC Single Cell Analysis Module is installed with access to a valid CLC Genomics Premium Modules license.

The results from these tools may be slightly altered as a result of the underlying improvement.

• Fixed an issue where the table view of a Venn Diagram could not be shown for data imported using the Import Expression Data tool of the Ingenuity Pathway Analysis plugin. The issue arose when a gene or transcript could not be matched to a location on the genome and the option “Unmatched genes/transcripts” was set to “Include”.

Improvements

• The RNA-Seq Analysis Report has been updated to include new text in the “Adapter read-through” section. The additional text explains that trimming the start of paired-end reads (5′ trim) can lead to spurious detection of adapter read-through.
• Update Sequence Attributes in Lists now accepts attribute information from comma separated (.csv) and tab separated (.tsv) format files.
• Volcano plots no longer include points where the fold change or y-axis component has an NaN value.
• When two or more reports produced by the same tool are used as input to Create Sample Report, and they contain conflicting values, the tool now fails. Previously, the results from the first report were used.
• When reports that contain identical tables and values are input to Create Sample Report, the values are included once in the report. Previously, a row per input report was included, even when the values were identical.
• Various minor improvements

Bug fixes

• Fixed an issue in Copy Number Variant Detection (CNVs), where a subset of region CNVs were not correctly calculated on chromosomes with low coverage target regions in the control samples. The issue led to overly large region CNVs. As gene-level CNVs are calculated from region-level CNVs, these were also affected. When only few target regions had low coverage in control samples, results were likely not affected.
• Fixed an issue that could the Workbench to crash on macOS Monterey when using the Welcome Center or viewing sunburst plots (which can be generated using tools in the CLC Microbial Genomics Analysis Module).
• Fixed an issue with the side panel settings where the label coloring of some restriction sites would always be black and the sorting would not be alphabetical when saved view settings were applied to a sequence.
• Fixed an issue where the wrong entry in a trim adapter list would be opened for editing if the list had been sorted or filtered.
• Fixed an issue that caused the Workbench to crash when installed on some Linux flavors if certain actions were taken and data compression was enabled.
• Fixed an issue in Maximum Likelihood Phylogeny that in rare situations led to tree construction never completing.
• Fixed an issue with the Create BLAST Database tool which could fail if the underlying native BLAST tool reported warnings.
• Fixed an issue where, when exporting elements to an S3 location in batch mode, the first element was exported successfully but the rest failed with an error.
• Fixed an issue with Split Sequence List when splitting based on attribute values, where an error resulted if the sequence list had more than 1000 distinct values for the selected attribute.
• Fixed an issue where a workflow could fail or produce an incorrect folder structure for outputs when it contained Split Sequence List, with splitting based on attribute values, and the name of at least one of the groups contained a forward slash “/” character.
• Fixed an issue affecting the naming of workflow outputs defined using a naming pattern of the form {input:N} or its equivalent {2:N}, e.g. {input:1} or {2:1}. The intended input name was not the one used to form the output element names when the workflow included an Iterate element, and import was done on the fly, and the batch units were defined based on the organization data, and:
  • Each batch unit was a set of paired reads, or
  • Each batch unit was a folder.
• Fixed an issue where the median read length in the supplementary report produced by QC for Sequencing Reads could be incorrect when the number of reads was very low. The median reported in the graphical report was correct.
• Fixed an issue with Create Sample Report affecting samples where reads mapped to only one chromosome. For such samples, the QC metric “% reads mapped in target region” was not included in the Quality Control table, even when that option had been selected when launching the tool.
• Various  minor bug fixes

Changes

The Java version bundled with CLC Genomics Server 22.0.1 is Java 11.0.14.1, where we use the JRE from the Azul OpenJDK builds.

New features and improvements

Template workflows

Template workflows are available from the Toolbox. These can be run as they are, or can be copied and customized for your needs. The template workflows provided are:

• Identify DNA Germline Variants – a workflow for detecting germline variants in DNA sequence data.
• RNA-Seq and Differential Gene Expression Analysis – a workflow for running RNA-Seq analyses on individual samples and then carrying out differential expression across specified groups represented by those samples.
• Prepare Raw Data – a workflow for quality reporting and adapter trimming of reads, intended for use before further analysis of that data.

Tools for annotating and manipulating sequence lists

The new Sequence Lists folder under Toolbox | Utility Tools contains tools for working with sequence lists. This includes existing tools, with new names and expanded functionality, as well as new tools:

• Split Sequence List New tool: Splits up nucleotide or peptide sequence lists. The output can be a specified number of lists, lists containing a specified number of sequences, or lists containing sequences with particular attribute values, such as terms in the description.
• Update Sequence Attributes in Lists New tool: Updates and adds information about the sequences in a list. For example, descriptions can be updated, or new information types can be added based on information  provided in an Excel file.
• Create Sequence List Existing tool. Create new sequence lists from sequence elements and/or sequence list elements. Previously available only from the File | New menu.

Access to Amazon S3 and BaseSpace

• Import data from Amazon S3 or BaseSpace when launching workflows
• Save workflow results to Amazon S3
• Access data in BaseSpace and configure a regional BaseSpace location

Other new functionality

• MGI/BGI importer An importer for MGI/BGI fastq format files.
• Rename Sequences in Lists Rename sequences within sequence lists by adding or removing characters, or replacing parts of names, optionally using regular expressions.
• Rename Elements  Rename elements by adding or removing characters, or replacing parts of names, optionally using regular expressions.
• A Heat map graphics exporter has been introduced for exporting heat maps to graphics file formats.
• Files containing tab separated values (.tsv) can be imported as tables using Standard Import.
• Export VDJ tools Exports T-Cell VDJ repertoire in txt format.

Improved menu organization and tool access

• New top level menus:
  • Connections For tools and functionality relevant to connections to other systems, such as a CLC Genomics Server.
  • Utilities For general tools and functionality such as search tools, the Plugin Manager,  Workflow Manager and Reference Data Manager.
• Improvements to the contents and order of tools in other top level menus
• The Favorites tab, where favorite tools and frequently used tools are listed for easy access, is now available in the Launch dialog and the workflow Add Elements dialog, in addition to in the Toolbox area in the bottom, left side of the Workbench.

RNA-Seq and Expression Analysis improvements

• Support for chimeric protocols has been improved in RNA-Seq Analysis.
• RNA-Seq Analysis now reports biotypes with frequency 0 in the “Distribution of biotypes” table.
• When included in workflows, PCA for RNA-Seq and Create Heat Map for RNA-Seq can be run using just one sample as input, thus enabling their use for both multi-sample and single-sample analyses.
• The statistical comparisons generated by Differential Expression for RNA-seq and Differential Expression in Two Groups includes the Biotype when it is available from the expression samples used as input.
• When using Create Expression Browser with miRNA input, the miRBase ID is preserved.
• When performing Differential Expression for RNA-seq on miRNA “group on mature” expression tables the miRBase ID is now exposed in the Statistical Comparison Tables.

Demultiplex Reads

• Demultiplex Reads now supports setting barcodes from table elements in addition to importing barcodes from local files.
• The barcode import table format has been extended to support additional columns.
• Barcode columns can be sorted.
• When running a workflow that contains a Demultiplex Reads element, the workflow wizard can calculate a preview and remove barcodes, similar to what is seen when running Demultiplex Reads directly from the Toolbox.
• When multiple elements are provided as input, the information in the Preview pane includes information obtained from across these. Previously, only the first input element was used for this.

BLAST related updates

• BLAST has been upgraded to BLAST+ 2.12.0 that includes a number of improvements and bug fixes. A full list of BLAST+ 2.12.0 changes can be viewed at http://www.ncbi.nlm.nih.gov/books/NBK131777.
• The list of databases available using BLAST at NCBI has been expanded, including the addition of ‘16S ribosomal RNA sequences (Bacteria and Archea)’ and ‘28S ribosomal RNA sequences from Fungi type and reference material (LSU)’.
• When BLAST at NCBI is used with multiple query sequences, the job will continue even if particular sequences fail due to a problem. Results for successful searches (including those with no hits) are returned. Sequences missing from the results due to problems are recorded in the job log.
• Searches against the Patented protein sequences database using BLAST at NCBI work once again. Previously, these searches always failed, with a dialog message saying only that no hits were found even though an error was returned by the NCBI. For affected searches, the error was reported in the job log.
• Fixed an issue affecting BLAST HSP Tables where the calculation of percent overlap between blast hits in reverse direction and query sequence was based on a sequence length that was 2 base pairs two short leading to incorrect values.
• Improvements have been made to make it less likely that a “CPU usage limit was exceeded” error will be returned when running blastp, blastx, tblastn or tblastx using BLAST at NCBI.

Importer and exporter improvements

• Multiple tables can be exported to a single file when using the following exporters: Tab delimited text, Annotation tab delimited text, Table CSV, Annotation CSV.
• A new custom reads option was added to the Illumina importer. The extended options for fastq file import has been added to support 10X data, it is e.g. now possible to import three fastq files with R1, R2, and I1 as paired reads where I1 is added in front of R1.
• When exporting variant tracks to VCF format, variants that fall under thresholds to be exported can now optionally be excluded entirely from the resulting VCF file.
• When using the VCF export setting for complex variant representation “Reference overlap and depth estimate”, complex overlapping reference alleles are now exported with a homozygous reference genotype.
• The list of supported GVF attributes in column 9 has been expanded when importing GVF files using the GFF2/GTF/GVF track importer.
• 1000 Genomes annotations are now better supported by the GFF2/GTF/GVF track importer.
• The Zygosity field is now included when exporting to GVF format.
• A subset of columns to export can be specified when exporting Mapping Coverage data.

Other improvements

• Copy Number Variant Detection (CNVs) can use coverage tables generated by QC for Targeted Sequencing as control mappings. Read mappings can still be used as control mappings.
• Copy Number Variant Detection (CNVs) allows different fold-change thresholds for deletions and amplifications.
• When working with paired reads, Trim Reads allows the trimming of a fixed number of bases to apply to only read 1, only read 2, or both reads of each pair.
• An option has been added to Extract Reads or Create Reads Track from Selection to allow just one member of a pair to be extracted when only one meets the extraction criteria.
• Extract Reads accepts stand alone read mappings in addition to reads tracks as input.
• Create Sample Report can take both the Graphical and the Supplementary Report created by QC for Sequencing Reads as input.
• An option has been added to Amino Acid Changes for using one letter amino acid codes in HGVS annotations.
• Filter on Custom Criteria now accepts expression tracks as input.
• In Quantify miRNA the option to select strand-specific analysis has been removed. The analysis is now always strand-specific.
• Remove Duplicate Mapped Reads considers if reads are duplicates based on the start position of reads instead of both start and end. This allows reads that have undergone quality trimming to be recognised as duplicates.
• The distance to consider around an intron-exon boundary when using Predict Splice Site Effect can be specified. Previously a length of 2 was always used.
• A choice of extinction coefficients has been introduced in Create Sequence Statistics.
• Create Mapping Graph can now generate graphs for forward read coverage and reverse read coverage.
• The Sample Reads tool is now named Subsample Sequence List and is located under the Utility Tools | Sequence Lists subfolder of the Toolbox. Peptide sequence lists are now accepted by this tool, in addition to nucleotide sequence lists. Existing workflows containing a Sample Reads element can be upgraded as normal. The element in the workflow will remain named “Sample Reads”, as seen in the Workflow Editor.
• In the tooltip displayed when moving the mouse cursor over reads in a fully zoomed-in view of a read mapping track, the number of reads supporting a deletion is displayed, in addition to the number of reads supporting particular base calls.
• When viewing data, tabs within the same tab area can be re-ordered by drag and drop.
• When a Track List and the tracks it refers to are copied in a single operation, the new copy of the Track List will refer to the the new copies of the tracks. Previously, the new Track List continued to refer to the original tracks.
• For workflows with paired read import as part of the workflow run, and when the workflow is launched in batch mode, or contains Iterate elements, paired read handling is now the same as for the relevant NGS importer tools (Illumina, Fasta, Sanger) themselves, irrespective of how batch units are defined or organized. Previously when batch units were based on data organization and all files were in the same folder, each file was treated as a separate batch unit irrespective of whether the Paired option was checked.
• In a workflow, Extract Annotated Regions (formerly Extract Annotations) can be connected to many more downstream tools than earlier.
• A Create Sequence List workflow element is available, replacing the New | Sequence List element. Create Sequence List can be connected to many more tools downstream than the earlier element.
• Memory usage when launching workflows in batch mode has been improved.
• Trim Sequences specifies which version of the UniVec database was used, both in the report and in the history of the trimmed sequences output.
• When the option to create a log is enabled when launching analyses in batch mode, a log file is created for each batch unit, as well as a combined log for all the analyses. Previously, only the combined log was generated.
• The table search criteria “is in list” and “is not in list” can be used with integers without specifying a thousand separators in the search term.
• The few tools that directly manipulate input elements, instead of generating a new element containing the changes as output, now generate a new element as output when used within a workflow. This allows them to be handled like any other tool in a workflow context.
• In addition to sequence elements, Add attB Sites accepts sequence lists with fewer than 10,000 sequences as input.
• Internal compression of CLC data has been improved. Elements created with this version of the software, with compression enabled, can be opened in version 21.0.5 and higher. Data must be exported or saved as uncompressed if sharing data with earlier versions of the software.
• Various minor improvements

Bug fixes

• Fixed an issue in Create Box Plot where percentiles reported in the history of a box plot element were off by one. For example, the “25%-ile” value was given the 24th percentile value. The correct values were used in the plots themselves.
• Fixed an issue in Demultiplex Reads where dual barcodes were not allowed to have mismatches in both barcodes.
• Fixed an issue in Demultiplex Reads where dual barcodes could previously be selected in random combinations. Dual barcodes are now handled in pairs.
• When using the “Genome annotated with genes only” in RNA-Seq Analysis, the range of annotation track types that can be used has been expanded. This includes the use of CDS annotation tracks, among others.
• Fixed an issue in Create Sample Report where, when QC thresholds had been specified for Trim Reads, wrong values from the Trim Reads report were shown in table 1.1 Quality Control of the sample report.
• Fixed an issue that caused Create Sample Report to fail when input reports did not contain values for specified QC thresholds.
• Fixed an issue in Combine Reports and Create Sample Report where the “Mean coverage per target” section would report coverages 10x too high when including a report from QC for Targeted Sequencing.
• Fixed an issue in VCF export where, in rare cases, variants below a specified minimum allele fraction threshold were not removed.
• Fixed an issue affecting Local Realignment where large indels upstream of a target region were sometimes not used when provided as guidance variants.
• Fixed an issue that in rare cases could cause Basic Variant Detection, Low Frequency Variant Detection and Fixed Ploidy Variant Detection to fail on very high coverage samples when the “Remove pyro-errors variants” option was enabled.
• Fixed an issue where Remove Duplicate Mapped Reads did not always de-duplicate paired reads with read-through correctly.
• Fixed an issue where Remove Duplicate Mapped Reads did not always de-duplicate reverse mapping single-end reads correctly.
• Fixed an issue affecting QC for Targeted Sequencing, where it failed with an error when an RNA-Seq read mapping containing paired reads was provided as input.
• Fixed an issue in Filter on Custom Criteria where numeric annotations were sometimes not allowed to be filtered using numerical operators such as “<“, “>”, “=”.
• Fixed an issue in Trio Analysis where, in rare cases, inconsistent zygosity between mother and father could lead to a wrong annotation of inheritance. Trio Analysis now reports inheritance as ‘Inconsistent zygosity’ if zygosity or the number of alleles is inconsistent between child, mother or father.
• Fixed an issue with VCF files exported from the CLC Genomics Workbench, where fusions that had one breakpoint in common were represented in a way that prevented QIAGEN Clinical Insight Interpret from displaying the counts.
• Fixed an issue causing Quantify miRNA to fail when there were empty entries in the Accession column of miRbase
• Fixed an issue causing BLAST hits with an identity below 40% to be shown in black even if the threshold for coloring was set lower than this.
• Fixed an issue where threshold values for color selectors in the side panel of the View Area could not be adjusted.
• Fixed an issue where specifying the color range values for heat maps in the side panel settings did not work.
• Fixed an issue where the names of outputs from Output elements attached directly to an Iterate element in workflows were not as intended when the metadata ({3} placeholder was used. We generally recommend that the specific input number(s) to include in output names are specified when configuring workflows that contain control flow elements.
• An element’s position within a folder in the Navigation Area can be controlled when copy/pasting, with the pasted element appearing above a selected element in the same folder. This fixes an issue introduced in CLC Genomics Workbench 12.0, where pasted elements were always placed at the bottom of the list in a folder when pasting.
• Fixed an issue where the content of the recycle bin was not shown correctly after the recycle bin had been emptied.
• Various bug fixes

Changes

• The Sample Reads tool is now named Subsample Sequence List and is located under the Utility Tools | Sequence Lists subfolder of the Toolbox. The functionality of this tool has been expanded. See the Improvements listing above, or refer to the manual.
• The Extract Annotations tool is now named Extract Annotated Regions.
• The tool Set Up Experiment is now named Set Up Microarray Experiment.
• The Track Tools folder, containing tools for working with track elements, has been moved from the top level of the Toolbox to under the Utility Tools folder.  Correspondingly, the workflow element for creating track lists is under the Utility Tools folder in the Add Elements dialog and no longer under the “New” list.
• The workflow element for creating sequence lists is under the Utility Tools folder. It no longer appears under the “New” list in the Add Elements dialog.
• The “Number of duplicates distribution” section has been removed from the report produced by Remove Duplicate Mapped Reads.
• When exporting BAM files, file names are limited to a maximum of 254 characters.
• Input modifying tools within workflows generate an output element instead of directly modifying the input provided. Workflows containing these tools may need to be edited.
• The Cut, Copy and Paste buttons have been removed from the toolbar. These functionalities are still available using items under the Edit menu or standard keyboard shortcuts.
• The Restore option under the Edit menu, for moving elements back out of the recycle bin, is now called Restore from Recycle bin.
• The Empty option under the Edit menu, for emptying the recycle bin, is now called Empty Recycle bin.
• The Java version bundled with CLC Genomics Workbench 22.0 is Java 11.0.10, where we use the JRE from AdoptOpenJDK.

Legacy tools

The following tools have been moved to the Legacy folder of the Workbench Toolbox and will be retired in a future version of the software:

• Batch Rename (legacy) This tool has been replaced by two new tools, Rename Elements, for renaming data elements, and Rename Sequences in Lists for renaming sequences within sequence lists.
• Empirical Analysis of DGE (legacy)

Functionality retirement

The following tools have been retired:

• Create Track from Experiment (legacy)
• Extract and Count (legacy)
• Download miRBase (legacy)
• Annotate and Merge Counts (legacy)
• Roche 454 NGxS import (legacy)
• Create Combined RNA-Seq Report (legacy)
• Remove Reference Variants (legacy)

The right-click option “Run in batch mode (legacy) for launching installed, multi-input workflows in Batch mode has been retired. Workflows can be launched in batch mode using standard launch functionality.