QIAGEN powered by

Latest improvements for QIAGEN CLC Genomics Server

  Current line         Previous line          Archive

QIAGEN CLC Genomics Server 21.0.3

Release date: 2021-01-26

Shared with workbenches

Bug fixes

  • Fixed an issue in that caused metadata layers to be displayed incorrectly on heatmaps produced by Create Heat Map for RNA-Seq. This issue affects analyses run using CLC Genomics Server 21.0.1 or 21.0.2, whether the tool is run independently, or included in workflows. We recommend deleting Heat Maps produced by affected software, and re-running the analyses. Please see the notification about this issue, which includes details about how to check if your results are affected.

Compatibility

The following are the corresponding clients for the QIAGEN CLC Genomics Server 21.0.3.

  • QIAGEN CLC Genomics Workbench 21.0.3
  • QIAGEN CLC Main Workbench 21.0.3
  • QIAGEN CLC Command Line Tools 21.0.3

We recommend running the corresponding versions of clients for QIAGEN CLC Genomics Server. However, QIAGEN CLC Genomics Workbench 21.0.1 and 21.0.2, QIAGEN CLC Main Workbench 21.0.1 and 21.0.2, and QIAGEN CLC Command Line Tools 21.0.1 and 21.0.2 can also connect to QIAGEN CLC Genomics Server 21.0.3.

CLC Server Command Line Tools

Compatibility

CLC Command Line Tools 21.0.3 is the corresponding client for QIAGEN CLC Genomics Server 21.0.3.

CLC Command Line Tools 21.0.3 can also act as a client for the QIAGEN CLC Genomics Server 21.0.1 and 21.0.1. However, we recommend running the corresponding version.



QIAGEN CLC Genomics Server 21.0.2

Release date: 2021-01-21

Shared with workbenches

Improvements and bug fixes

  • Fixed an issue where Demultiplex Reads run within a workflow context could not be run on paired reads where the barcode was defined on the mate.
  • Various minor improvements

Compatibility

The following are the corresponding clients for the QIAGEN CLC Genomics Server 21.0.2.

  • QIAGEN CLC Genomics Workbench 21.0.2
  • QIAGEN CLC Main Workbench 21.0.2
  • QIAGEN CLC Command Line Tools 21.0.2

We recommend running the corresponding versions of clients for QIAGEN CLC Genomics Server. However, QIAGEN CLC Genomics Workbench 21.0.1, QIAGEN CLC Main Workbench 21.0.1, and QIAGEN CLC Command Line Tools 21.0.1 can also connect to QIAGEN CLC Genomics Server 21.0.1.

CLC Server Command Line Tools

Bug fixes

  • Fixed an issue where using the -I argument could result in an error if it was not the last argument provided in the command.

Compatibility

CLC Command Line Tools 21.0.2 is the corresponding client for QIAGEN CLC Genomics Server 21.0.2. It can also act as a client for the QIAGEN CLC Genomics Server 21.0.1. We recommend running the corresponding version of the CLC Command Line Tools CLC Genomics Server.



QIAGEN CLC Genomics Server 21.0.1

Release date: 2021-01-12

This is a compatibility release for the corresponding client software, QIAGEN CLC Genomics Workbench 21.0.1 and QIAGEN CLC Main Workbench 21.0.1.

Please see the release notes for CLC Genomics Server 21.0, below, for a full list of changes since the last general release of this software.

Compatibility

The following are the corresponding clients for the QIAGEN CLC Genomics Server 21.0.1.

  • QIAGEN CLC Genomics Workbench 21.0.1
  • QIAGEN CLC Main Workbench 21.0.1
  • QIAGEN CLC Command Line Tools 21.0.1

We recommend running the corresponding versions of clients for QIAGEN CLC Genomics Server. However, QIAGEN CLC Genomics Workbench 21.0, QIAGEN CLC Main Workbench 21.0, and QIAGEN CLC Command Line Tools 21.0 can also connect to QIAGEN CLC Genomics Server 21.0.1.

CLC Server Command Line Tools 21.0.1

This is a compatibility release, as the corresponding client for the QIAGEN CLC Genomics Server 21.0.1.

Please see the release notes for CLC Server Command Line Tools 21.0, below, for a full list of changes since the last general release of this software.

Compatibility

CLC Command Line Tools 21.0.1 is the corresponding client for QIAGEN CLC Genomics Server 21.0.1. It can also act as a client for the QIAGEN CLC Genomics Server 21.0. We recommend running the corresponding version of the CLC Command Line Tools CLC Genomics Server.



QIAGEN CLC Genomics Server 21.0

Release date: 2021-01-12

Server specific

New features

Plugins and licenses can be downloaded via the web administrative interface

Software licenses and server plugins can be downloaded and updated directly via the web administrative interface, under the new Extensions tab. In association with this, plugin management has been moved to the new Extensions tab, and the display of information about installed plugins has been improved.

Genomics Analysis Portal

The Genomics Analysis Portal is a web browser based, graphical client for the the CLC Genomics Server. Workflows can be submitted for analysis on the server from a web browser, analysis progress and status of analyses can be monitored, and results can be downloaded.

Enabling the Genomics Analysis Portal is described in the CLC Server Admin manual, while configuration and use is described in the Genomics Analysis Portal manual.

“Containerized” External applications

Portable, “containerized” external applications using Docker are supported on Linux-based CLC Server setups. Containers can be local or in a repository, such as Amazon Elastic Container Repository (AWS ECR) or Docker Hub. Like standard external applications, they can be run as individual tools or included in workflows.

User folders for user-level permissions

One or more File system locations can be configured to be used for “user home” folders, which are top-level folders within that location. A user is granted write access only to the folder with a name matching their username. These “user home” folders can be created automatically on user login, or created manually for select users.

Improvements

External applications
  • Support has been added for automatic update of external applications.
  • Tooltips, visible in the launch wizard, can be added for external applications parameters.
  • The order that parameters are displayed in the Workbench launch wizard reflects the order in the external application configuration. Previously the order in the launch wizard reflected the time the parameters were added.
  • The order of parameters in an external application can be rearranged directly in the configuration editor. Previously, the parameter needed to be deleted and then reinserted and reconfigured.
  • The namea of “standard out” and “standard error” to be shown in workflows, as well as the name of the files saved, can be configured.
  • Improved presentation of parameter settings in the General configuration tab.
  • The folder-level organization of external applications is reflected in the Export, Delete and Publish dialogs.
General
  • The location that intermediate workflow results should be stored can now be configured. They can be stored in a temporary directory of the system the workflow is executed on or stored in a subfolder of the location that final results will be stored. The default is the latter, which is also the behavior of earlier CLC Genomics Server versions.

Bug fixes

  • Fixed an issue where output naming patterns referring to specific inputs (such as {input:2}) produced default output names instead of the expected substituted ones. This issue arose for some workflows containing Iterate control flow elements when run on a CLC Genomics Server configured to “Submit tasks in each workflow block to a single node” (Workflow queuing option).

Changes

  • The default setting for “Workflow queuing options” has been changed to to “Submit tasks in each workflow block to a single node”.

Shared with CLC Workbenches

New features and improvements

Full workflow support for Sanger sequence analysis

New features have been introduced, and improvements made, to support automated analyses of Sanger trace data using workflows.

Trim Sequences
  • Trim Sequences is available on the CLC Genomics Server.
  • Trim Sequences can be used in workflows.
  • A new sequence element containing the trimmed sequences is output. Previously, the input was modified and saved.
  • A report can be generated containing a summary of the number of reads trimmed and the reasons for the trimming. This report is supported by the Combine Reports tool.
  • The UniVec database used in this tool has been updated to version 10.0 of UniVec_Core.
Other improvements supporting trace data analysis in workflows
  • Trace data can be imported using on-the-fly import in workflows.
  • Improved output naming by the Assemble Sequences to Reference and Assemble Sequences tools: The sample name is included in the file name and the sequence names in the output.
  • Metadata-based naming is supported in workflows run in batch mode or with Iterate control flow elements through the use of new placeholders: {metadata} and {metadata:<columnname>}.
  • The Secondary Peak Calling tool no longer modifies the input data element, but instead produces new elements as output. Note: This change requires that the workflows with this tool that were created in older versions of the software must be manually updated. The old workflow element must be replaced by a new one. The recommended upgrade path for installed workflows containing the Secondary Peak Calling tool is to save a copy of the workflow in the Navigation Area using a version 20.x Workbench, and then open and manually update that workflow in the CLC Genomics Workbench 21.0. or CLC Main Workbench 21.0. The new workflow can then be installed, if desired.
New tools
  • Create Sample Report creates a summary report of selected information from multiple reports relating to a single sample. Specific types of information can be specified for inclusion in the Quality Control section.
  • Extract IsomiR Counts extracts information from the read mappings of each miRNA or other custom added database type, e.g. piRNA etc, and collects the information across all mappings in a table that can be exported.
  • Annotate with Repeat and Homopolymer Information adds annotations to variants by appending two new columns with information about repeat and homopolymer status.
  • Merge Variant Tracks merges multiple variant tracks into a single track. Options are available for appending annotations from overlapping variants.

Extract IsomiR Counts, Annotate with Repeat and Homopolymer Information and Merge Variant Tracks were previously available via the Biomedical Genomics Analysis Server Plugin.

Workflow related
  • When a workflow with Export elements is run in batch mode, the exported files from each batch run can be saved to separate folders.
  • BED and VCF format files can be imported on-the-fly in workflows.
  • On-the-fly import can be used without metadata when running workflows in batch mode, and when running workflows containing a single Iterate element.
  • Name placeholders for output elements and export elements have been updated, and the naming of outputs of workflows run in batch mode can be more finely controlled.
  • Improvements for Workflow Input elements
    • Workflow Input elements can be configured to limit the data input method to either selection of data elements from the Navigation Area or selection of files to be imported using on-the-fly import. The default is to allow the input method to be chosen when launching the workflow.
    • Workflow Input elements can be configured to limit the on-the-fly import types available when launching the workflow. Parameters of selected importers can also be locked or unlocked, as desired, defining whether the setting is configurable when launching the workflow.
  • Additional configuration options for Iterate and Collect and Distribute workflow elements are available.
  • When a workflow with Iterate elements is run with the “Batch” checkbox checked, the “Batch identifier” column in the Workflow Result Metadata table will contain the combined batch identifier, reflecting all levels of batching and iterations.
  • The following tools are available to be included in workflows:
Performance improvements
Export
  • Exported files can be saved into subfolders of the selected output area by using a forward slash character / at the start of the custom file name definition.
  • Graphics export of Tracks, Track lists, Sequences, Alignments and Read mappings is supported as a standard export, which can be embedded into workflows and executed on a CLC Genomics Server. This feature is intended for high-throughput applications. This feature is intended for high-throughput applications.
  • The naming pattern for files exported using the fastq exporter has been updated to be in line with the naming format the Illumina importer expects. The exported file names now end with “_R1.fastq” and “_R2.fastq”. Previously the extension used was “.R1.fastq” when exporting a single file, if pairs where exported to two files, the second file had the extension “.R2.fastq”. (The first “.” in the original naming has been replaced by an “_”).
  • Export VCF has been updated:
    • It supports the export of CNV and fusion data.
    • If multiple elements have been selected for export, there is an option for exporting them to a single file.
    • It uses the value “.” to represent missing variant annotations.
    • Special characters in variant annotations are exported using percent encoding, as specified in VCF 4.3.
Illumina importer
  • The “Paired reads” option is enabled by default.
  • Improved validation when the “Paired reads” option is enabled,. The names of the pairs of files are validated as follows:
    • If the file names follow the Illumina naming format, the two files are required to have the same sample name and lane
    • If the file names do not follow the Illumina naming format, but _R1/_R2 is detected in the names, the first file must contain _R1 and the second file must contain _R2.
    • If the “Join reads from different lanes” option is enabled, the detected lane, in the format _L001, must be the same for both files.
    • If a pair of files does not meet the requirements above, a message is printed in the log and the pair of files is skipped.
  • Improved naming of the imported elements:
    • If the imported files follow the Illumina naming format, the imported elements no longer contain the _R1_001 suffix.
    • Otherwise, if _R1 / _R2 is detected in the names of the files, it is removed from the name of the imported elements.
Local Realignment
  • A restriction has been removed from Local Realignment that prevented paired reads from being realigned when that realignment would change which read was left-most on the reference. The overall effect of this change is to increase the likelihood of detecting insertions in rare cases.
  • Improvements have been made when realigning large insertions at the beginning of reads.
  • The “Allow guidance insertion mismatches” and “Maximum guidance-variant length” options are now enabled only when a guidance-variant track is provided.
  • Fixed an issue that caused reads with unaligned ends stretching over a chromosome boundary to be removed from the mapping.
  • Local Realignment respects the CPU limit defined via the web administrative interface, if a limit has been set.
QC for Targeted Sequencing
  • A new option in QC for Targeted Sequencing allows a custom list of coverage levels to be specified.
  • The report includes the complete set of chromosomes in the “Targeted region overview” section when using references with up to 200 chromosomes. Previously the limit was 100 chromosomes. This change means the hg38_no_alt_analysis_set reference data set, available from the Reference Data Manager, is now supported.
  • The report has been extended with values reporting the number and percentage of base positions in target regions with coverage above or equal to the minimum threshold.
Other improvements
  • Improved the alignment quality for read mappings by removing aligned ends with an alignment score of zero. As a result, some alignments will be shorter and may be filtered away because they no longer pass the minimum length fraction criterion. Tools benefiting from this change include Map Reads to Reference, RNA-Seq Analysis, Map Reads to Contigs and Map Bisulfite Reads to Reference.
  • Option names and other information in the wizards for the Trim Reads tool and the corresponding workflow element have been updated for clarity and consistency.
  • De Novo Assembly reports can be used as inputs to the Combine Reports tool.
  • A new option, “Filter on average expression for FDR correction” is available in Differential Expression for RNA-Seq and Differential Expression in Two Groups. When checked, automatic, independent filtering prior to FDR correction is carried out, with the aim of increasing power.
  • Plots and tables generated by QC for Sequencing Reads have better usability, especially when working with long reads. Tables with more than 500 data points now show the first 100 entries and then bin remaining data points, based on range. In graphs, end positions with a coverage below 0.005% across the reads are not included.
  • QC for Sequencing Reads reports the QC metrics separately for different read types found in the input data: unpaired reads, R1 reads and R2 reads.
  • In Quantify miRNA, the minimum value for the setting “Minimum sequence length”, used for seed counting, has been changed to 8. (The seed is a 7 nucleotide sequence from positions 2-8 on the mature miRNA.)
  • The Quantify miRNA outputs, “Grouped on mature” and “Grouped on seed tables”, contain links to miRBase.
  • A new section has been added to the Call Methylation Levels report containing details of read conversion and direction.
  • Remove Duplicate Mapped Reads outputs reads in a deterministic order.
  • The “Reads trimmed (%)” column in the “Trim summary” section of the Combine Reports output has been removed as it was a duplicate of the “Reads after trim (%)” column.
  • Custom attributes can be configured in a data location such that attribute values are not copied when copying data elements.
  • Annotate with Overlap Information and Filter Based on Overlap now count insertions and zero-length annotations as overlapping a region when they overlap either border. E.g. when an insertion is right on the border of a gene, we say that the insertion overlaps the gene.
  • The SRA toolkit has been updated to version 2.10.7.
  • Various minor improvements

Bug fixes

  • Fixed an issue affecting Filter on Custom Criteria when included in a workflow with the filtering step option unlocked. If criteria were updated, added, or removed filter in the launch wizard, the updated criteria were not used in the first run of the workflow with these updated values. Instead, the old criteria were used in that run. In subsequent runs, the updated values were used.
  • Fixed an issue affecting read mappings where a short deletion was preferred to a mismatch for equal scoring alignments. Tools benefiting from this change include
  • Map Reads to Reference, RNA-Seq Analysis, Map Reads to Contigs and Map Bisulfite Reads to Reference.
  • Fixed an issue in Trim Reads where length filters were applied before automatic read-through adapter trimming was done, if it was enabled. This could result in reads shorter than minimum length settings being included in the output.
  • Fixed an issue affecting Basic Variant Detection, Fixed Ploidy Variant Detection and Low Frequency Variant Detection, where forward coverage or reverse coverage could be reported as being higher than it was when looking for very low frequency variants with very low minimum count values.
  • Fixed a bug where IonTorrent SAM files with special characters in the sample name could not be imported in separate folders.
  • Fixed an issue where Map Reads to Reference could occasionally ignore reads when encountering a read with an unaligned end that wraps twice around a chromosome.
  • Fixed an issue in Quantify miRNA where the isomiRs associated with a reference mir-rna were not all consistently named using the miRbase isomiR nomenclature (http://www.mirbase.org/help/nomenclature.shtml).
  • Fixed an issue in Create Heat Map for RNA-Seq affecting the “Fixed number of features” option, where one member of the set of most variable genes or transcripts was missing from those used in the analysis, with a slightly less variable feature included instead.
  • Fixed an issue in Create Heat Map for RNA-Seq, where the “Filter by statistics” option could not be used with miRNA expression data.
  • Fixed an issue in Create Heat Map for RNA-Seq, where the history of heat maps did not include the name or the version of the tool used.
  • Fixed an issue where RNA-Seq Analysis failed if a read mapped across 2 exons of a gene, where those 2 exons spanned the origin of a chromosome.
  • Fixed an issue where RNA-Seq Analysis failed if a gene or mRNA spanned the origin of a chromosome and that chromosome was marked as linear. We now ignore these mRNAs.
  • Fixed an extremely rare issue where RNA-Seq Analysis could fail when the positions of genes (or transcripts) were defined with respect to a sequence that was not part of the genome. An example of this kind of annotation is the remote entry identifier allowed by GenBank flat file format, see http://www.insdc.org/files/feature_table.html#3.4 These genes and transcripts are now filtered away prior to the tool being run.
  • Fixed an issue that caused Combine Reports to occasionally fail when combining reports with summary information shown as plots.
  • Fixed an issue with Combine Reports where, when combining RNA-Seq reports, warning messages for the “Distribution of biotypes” section could be present when they should not have been.
  • Fixed an issue where a wrongly formatted VCF file could make the VCF importer terminate instead of writing the error to the log.
  • Fixed an issue where Transcription Factor ChIP-Seq would exit with an error when given a read mapping with a circular reference sequence with coverage across all bases.
  • Fixed an issue affecting the Basic Variant Detection, Fixed Ploidy Variant Detection and Low Frequency Variant Detection tools, where complex indels were reported in regions where the reference had a sequences of Ns. This error was introduced in CLC Genomics Server 20.0.2.
  • Fixed an issue that could cause De Novo Assembly to occasionally fail when assembling paired data with both the "Auto detect paired distances" and ""Map reads back to contigs (slow)" options enabled.
  • Fixed an issue with links to HGNC in gene tracks imported from GFF3 files using “Import Tracks from File” and in some Refseq gene tracks provided via the Reference Data Manager.
  • Fixed an issue in Excel importer, where the presence of certain formulas would previously prevent successful import.
  • Fixed an issue where, if BLAST at NCBI failed with an error, no error would be shown and instead no hits were returned.
  • Fixed a bug where some workflows using a Collect and Distribute element with multiple output channels did not pass the correct inputs to a tool after the Collect and Distribute element
  • Various minor bug fixes

Changes

  • The Java version bundled with CLC Genomics Server 21.0 is Java 11.08, where we use the JRE from AdoptOpenJDK.
  • The read mapping tool used by various tools in the CLC Genomics Server (e.g. Map Reads to Reference, RNA-Seq Analysis, Map Reads to Contigs and Map Bisulfite Reads to Reference) has been updated for this release and corresponds to the version in CLC Assembly Cell 5.2.1. Other binaries are unchanged and continue to correspond to the versions in CLC Assembly Cell 5.1.1.
  • The default base name for the element being exported is designated using the placeholder {name}, instead of {input}. The numeric equivalent, {1}, is unchanged. The default export naming pattern has correspondingly been changed to {name}.{extension}. (GxS notes only, add the following: This change also applies to exports configured in External Applications.) Previously {input} was used.
  • The default expect value (e-value) for BLAST at NCBI is 0.05 and the maximum number of hits is 5000, aligning with the defaults used at the NCBI.
  • Changes have been made to the handling of sequence identifiers when using Create BLAST Database. This change allows continued flexibility in the naming of sequences used for making these databases, avoiding direct exposure to limitations present in the underlying BLAST+ program, makeblastdb, such as not allowing long or duplicate sequence names. Further details are provided in our FAQ area.
  • The option “Reports originate from a single sample” has been removed from the Combine Reports tool. For generation of a single sample combined report, please use the new Create Sample Report tool.
  • The “Chromosome M name” option in Trio Analysis has been renamed to “Chromosome MT name”, with default value “MT” instead of “M”.
  • The creation of Workflow Result Metadata tables is optional when running workflows on the CLC Genomics Server.

Functionality retirement

Tools

  • Reverse Sequence

Compatibility

The follow are the corresponding client applications for CLC Genomics Server 21.0

      • CLC Genomics Workbench 21.0
      • CLC Main Workbench 21.0
      • CLC Command Line Tools 21.0

CLC Genomics Server 21.0 is compatible with GCE version 21.0.

Plugin notes

CLC Server Command Line Tools 21.0

These are the draft release notes for CLC Server Command Line Tools 21.0, due for release on January 12, 2021.

The draft manual is available in PDF format and HTML format.

Installers for this product are available as “early access” via links at the bottom of this page. These products are not supported, and we recommend they are not used in production during the early access period.

New features

  • -Y Include this option in a command to run it asynchronously
  • -I Get information about particular processes or to list all processes submitted by the user running the command.
  • -R Get the results of finished processes or cancel processes.

New tools

Analysis related
  • anno_with_repeat_and_homopoly_info
  • create_sample_report
  • extract_isomir_counts
  • merge_variant_tracks
  • trim_sequences
New exporters
  • alignment_graphics
  • mapping_graphics
  • sequence_graphics
  • track_graphics
  • track_list_graphics
  • trim_sequences
New importers
  • trace_files_import
Utility tools
  • request_install_server_license
  • user_home_read_settings
  • user_home_write_setting

New and updated options for existing tools

Analysis related
  • process_tagged_sequences – new options added
    –aux-values
    –barcode-columns
    –barcode-table-file–mapping-source-type
    –name-column
  • statistics_target_regions – new options added
    –coverage-levels replaces the old option –report-type, which has been removed
    –custom-coverage-levels
  • For export of vcf
    • –onefile <Boolean>
Utility tools
Options added
  • rm – new option added
    –direct

 

Options removed
  • combine_reports
    –single-sample (The new create_sample_report tool caters for this situation)

 

Changes
  • ngs_import_illumina The default for the –paired-reads option is now true. It was previously false.

Commands removed

  • reverse_sequence

 

Bugfixes

  • Fixed an issue that caused information about libraries intended for the LICENSE and NOTICE files to be omitted.

Other changes

  • The Java version bundled with CLC Server Command Line Tools 21.0 is Java 11.08, where we use the JRE from AdoptOpenJDK.