QIAGEN powered by

Latest improvements for QIAGEN CLC Genomics Cloud Engine

  Current line         Previous line          Archive

QIAGEN CLC Genomics Cloud Engine 21.0.3

Release date: 2021-05-25

This update contains only changes to the software. There are no changes to the infrastructure configuration.

Improvements

  • Tokyo (ap-northeast-1) AWS region is now supported.
  • The internal file system used has been changed from XFS to EXT4.

Changes

  • CLC Genomics Cloud Engine 21.0.3 supports workflows launched using CLC Genomics Workbench 21.0.4 and CLC Main Workbench 21.0.4, as well as earlier Workbench versions in the 21 release line.

Frequently Asked Questions

CLC Genomics Cloud Engine Command Line Tools 21.0.3

This is a compatibility release, as the corresponding client for the CLC Genomics Cloud Engine 21.0.3.

Compatibility

Workflow compatibility

Workflows to be run on the CLC Genomics Cloud Engine 21.0.3 are designed using a CLC Workbench. The recommended versions for this purpose for this version are:

  • CLC Genomics Workbench 21.0.4
  • CLC Main Workbench 21.0.4

CLC Genomics Cloud Engine 21.0.3 contains CLC Genomics Server 21.0.1, 21.0.2, 21.0.3 and 21.0.4. Workflows can be  submitted for execution using a corresponding client Workbench version with the Cloud Plugin installed. Workflows can also be submitted via corresponding versions of the CLC Genomics Server with the Cloud Server Plugin installed.

Service/Client compatibility

The CLC Genomics Cloud Engine Command Line Tools 21.0.3 is the corresponding client for the CLC Genomics Cloud Engine 21.0.3.

While we recommend running corresponding versions of service and client, CLC Genomics Cloud Engine Command Line Tools 21.0.3 can be used with earlier CLC Genomics Cloud Engine 21.0.x versions, and CLC Genomics Cloud Engine Command Line Tools 21.0.3, can be used with earlier CLC Genomics Cloud Engine 21.0.x versions.



QIAGEN CLC Genomics Cloud Engine 21.0.2

Release date: 2021-04-19

This update contains only changes to the software. There are no changes to the infrastructure configuration.

Improvements

  • Updates were made to accommodate changes at AWS. There are no changes to product functionality.

Frequently Asked Questions

CLC Genomics Cloud Engine Command Line Tools 21.0.2

This is a compatibility release, as the corresponding client for the CLC Genomics Cloud Engine 21.0.2.

Compatibility

Workflow compatibility

Workflows to be run on the CLC Genomics Cloud Engine 21.0.2 are designed using a CLC Workbench. The recommended versions for this purpose for this version are:

  • CLC Genomics Workbench 21.0.3
  • CLC Main Workbench 21.0.3

CLC Genomics Cloud Engine 21.0.2 contains CLC Genomics Server 21.0.1, 21.0.2,  and 21.0.3. Workflows can be  submitted for execution using a corresponding client Workbench version with the Cloud Plugin installed. Workflows can also be submitted via corresponding versions of the CLC Genomics Server with the Cloud Server Plugin installed.

Service/Client compatibility

The CLC Genomics Cloud Engine Command Line Tools 21.0.2 is the corresponding client for the CLC Genomics Cloud Engine 21.0.2.

While we recommend running corresponding versions of service and client, CLC Genomics Cloud Engine Command Line Tools 21.0 can be used with CLC Genomics Cloud Engine 21.0.2, and CLC Genomics Cloud Engine Command Line Tools 21.0.2, can be used with CLC Genomics Cloud Engine 21.0.



QIAGEN CLC Genomics Cloud Engine 21.0.0

Release date: 2021-01-12

This update contains changes to the infrastructure as well as the software. Because of the infrastructure changes, it is necessary to redeploy the CLC Genomics Cloud Engine.

Please also refer to the Cloud Plugin 21.0 latest improvements for associated features and improvements affecting the connection to and use of CLC Genomics Cloud Engine 21.0 from CLC Genomics Workbench 21.x and CLC Genomics Server 21.x software.

New Features

  • Multiple AWS EC2 instance types are supported. A default set comes preconfigured, but this list can be adjusted in the admin host. Instance types are no longer managed in the Elastic Beanstalk service.
  • When workflows containing tools distributed via plugins are run, the relevant plugin version is dynamically deployed to GCE.
  • Data stored in Illumina BaseSpace can be imported directly using on-the-fly import in workflows.
  • Third party software can be executed from within workflows run on GCE. This is done by including elements in the workflow for containerized external applications that have been made available in Amazon ECR. A local CLC Genomics Server is required for configuring such external applications.
  • GCE releases from this point forward are backward compatible with earlier CLC Genomics Server versions within the same major release line.

Improvements

Performance improvements

  • Substantially improved memory use (up to 4 times less memory required for each job)
  • Executor instances start up faster (up to 3 times faster than previously)
  • Executor instances shut down faster

Integration improvements

  • Events are sent when a batch is complete, not just when a job completes.

Other improvements

  • Additional log files from the CLC Genomics Server have been added to CloudWatch.
  • Default log retention has been set to 180 days.
  • The Spring property: “security.oauth2.resource.id” can be used for OAuth2 configuration.
  • Cache buckets can be created in regions and AWS accounts other than those the GCE is deployed in.
  • The form filled out when creating a cache bucket using CloudFormation has been simplified.
  • The job status description has been improved for situations where a job is rescheduled due to a timeout.
  • If a job is submitted for a workflow created in a CLC Workbench version that is not compatible with a version of the CLC Genomics Server available in the GCE instance, a validation error is reported to the user and the job is not submitted.

Bug fixes

  • Fixed an issue that caused an error when all plugins were deleted.

Changes

  • The Java version bundled with CLC Genomics Cloud Engine 21.0 is Java 11, where we use the JRE from Amazon Correto.
  • Job priority options were replaced by option to specify instance types.
  • Several scripts on the admin host have been renamed or moved. The CLC Genomics Cloud Engine administrator manual contains the up-to-date script names. The changes include, but are not limited to:
    • Prepare script: admin/prepare.sh → ./prepare.py
    • Install GCE: ./create-stack.py → ./create_stack.py
    • Configure GCE stack name: /root/setup.sh → /root/config/setup.sh
    • Configure authentication: ~/job-manager-config/application-oauth2.yaml → ~/config/application-oauth2.yaml
    • Restart license server: admin/tools/restart-licenseserver.sh → ./restart_license_server.py
  • The CloudFormation form used when creating an admin host no longer requires the definition of a solution stack.
  • The admin host no longer uses the Elastic Beanstalk service.

Frequently Asked Questions

CLC Genomics Cloud Engine Command Line Tools 21.0.0

Bug fixes

  • Fixed an issue that caused job submissions to fail if the workflow .cpw file was located in a bucket with a name that contained a period.

Compatibility

The CLC Genomics Cloud Engine Command Line Tools 21.0.0 is the corresponding client for the CLC Genomics Cloud Engine 21.0.0.

It is not possible to access the CLC Genomics Cloud Engine 21.0.0 with an earlier version of the CLC Genomics Cloud Engine Command Line Tools. Nor is it possible to access an earlier version of the CLC Genomics Cloud Engine with the CLC Genomics Cloud Engine Command Line Tools 21.0.0.

The CLC Genomics Cloud Engine uses CLC Genomics Server 21.0.1.

Workflows to be run on the CLC Genomics Cloud Engine 21.0.0 are designed using a CLC Workbench. Workflow installer files should be made using a CLC Workbench compatible with the CLC Genomics Server run by GCE. For this release, these are:

  • CLC Genomics Workbench 21.0.1
  • CLC Main Workbench 21.0.1

Workflows can be submitted to run on CLC Genomics Cloud Engine 21.0.0 from compatible versions of CLC Workbenches with the Cloud Plugin installed and from compatible versions of CLC Genomics Server with the Cloud Server Plugin installed.