This update contains changes to the infrastructure as well as the software. Because of the infrastructure changes, it is necessary to redeploy the CLC Genomics Cloud Engine.
- Batch submission is now supported, allowing CLC GCE users to submit large number of jobs in a single request.
- Files can now be accessed using internal Amazon S3 protocols, which is provided free of charge by AWS. Accessing files via pre-signed URLs is still possible, and should be used when user access rights need to be controlled.
- The distribution of jobs can now be stopped by the administrator. This allows maintenance tasks like changing instance type to be undertaken without stopping job submissions.
- Amazon S3 cache deployment scripts have been introduced. These create one or more cache buckets and are needed to support submission of jobs from CLC Workbenches with the Cloud Plugin installed, or via a CLC Genomics Server with the Cloud Server Plugin installed. These plugins are due for release soon.>
- The OAuth2 Client Credentials flow authentication protocol is now supported.
- Job and Batch search functionality are now available via the REST API.
- The validation of job and batch submissions has been substantially improved. This includes validating the version of the CLC Workbench and any associated plugins used to create the workflow.
- A job is no longer launched on GCE in response to requests querying the parameters of a workflow.
- The version of the CLC Genomics Server being used by CLC GCE can now be inspected via the description of the Elastic Beanstalk application version.
- The deleteStack.sh script no longer tries to delete the certificate associated with the JobManager Load balancer.
- All node.js scripts have been upgraded from version 8 to version 10.
- CLC GCE now places results files in subfolders with names that include the job name as well as a unique job ID.
- The maximum number of jobs in a JobManager database has been increased from 100,000 to 10,000,000.
- Fixed an issue that could sometimes result in QIAGEN reference data not being put in the expected location when downloaded automatically when launching a workflow on the CLC Genomics Cloud Engine. When this happened, workflows depending on that reference data would fail to run to completion. This problem affected CLC Genomics Cloud Engines running CLC Genomics Server 11.0.
- Fixed a timeout error that occurred on the JobManager web pages. Users are now presented with a login dialog if the login has timed out.
Frequently Asked Questions
- A new End User License Agreement (EULA) has been introduced.
- Multiple jobs can now be submitted with a single command as a batch.
- Querying the parameters of a workflow is now much faster.
- Data from AWS regions other than the one the CLC GCE instance being used is installed upon can now be used.
- GCE CLT no longer pre-signs S3 URLs unless it is is needed for CLC GCE to access the data and automatic pre-signing of S3 URLs has not been disabled in the GCE CLT configuration.
- Instance type and disk size can no longer be changed using GCE CLT. Disk size now scales automatically as needed. Instance type can be changed via the AWS Elastic Beanstalk web console.
The CLC Genomics Cloud Engine Command Line Tools 20.0.10 is the corresponding client for the CLC Genomics Cloud Engine 20.0.10.
We always recommend running the corresponding versions of service and client and for this release in particular it is not possible to access the CLC Genomics Cloud Engine 20.0.10 with an earlier version of the CLC Genomics Cloud Engine Command Line Tools. Nor is it possible to access an earlier version of the CLC Genomics Cloud Engine with the CLC Genomics Cloud Engine Command Line Tools 20.0.10.
The CLC Genomics Cloud Engine uses CLC Genomics Server 20.0.1.
Workflows to be run on the CLC Genomics Cloud Engine 20.0.10 are designed using a CLC Workbench. Workflow installer files should be made using a CLC Workbench compatible with the CLC Genomics Server run by GCE. The recommended versions for this purpose for this version are:
- CLC Genomics Workbench 20.0.1
- CLC Main Workbench 20.0.1
Workflows can be submitted to run on CLC Genomics Cloud Engine 20.0.10 from compatible versions of CLC Workbenches with the Cloud Plugin installed and from compatible versions of CLC Genomics Server with the Cloud Server Plugin installed.