The key to understanding HybridCluster is that it is the integration of four fundamental components and the way that these components combine together that creates a system that provides high availability, auto-scaling, and data vault capabilities all in a single deployable package.

It’s not magic, it’s just beautifully architected to solve the problems that hosting companies and cloud providers face. And it’s based on technology that you can trust – Apache, MySQL, FreeBSD, ZFS, Twisted.

A cluster consists of at least two servers connected via public or private networks:

hybrid-cluster-global-diagram

 

Each node consists of the following components:
System Overview Diagram

AwesomeProxy

A distributed proxy server, that handles HTTP(S), (S)FTP, MySQL, POP, IMAP, SMTP (+ STARTTLS), SSH and even generic TCP requests and can buffer and route them across and between machines.

FlexContainers

Use lightweight OS virtualisation techniques (e.g. BSD Jails and chroot style containers) to isolate processes, enabling high density secure operation with minimal overhead.

Storage

A distributed Storage system, harnessing the power of ZFS, that creates and manages all the file systems across all the machines and implements file system versioning and ultra-fast snapshotting.

Control and Management

A set of distributed Control and Management algorithms that orchestrates the behaviour of these components by:

  • Instrumenting and Monitoring all the servers and connections between them
  • Managing the FlexContainers and their data (e.g. mailboxes, websites, databases and jails), and controlling data replication and snapshotting
  • Implementing the core algorithms to balance load and recover from errors using the snapshot graph data, load data and heartbeat data that it has assembled
  • Providing billing APIs to enable integration with industry standard billing engines such as WHMCS
  • Providing UIs for system administrators and users via cPanel-like interfaces that give comprehensive management and control

For more detail on the way these core components are used to create the whole system, see High AvailabilityAuto-scaling, and Data Vault.

There is also a centralised deployment system which provides an API which integrate with EC2/ElasticHosts/SoftLayer to enable direct integration with public cloud services to make it easy to spin up clusters which host applications and services which span local and cloud infrastructure.

— Back to top —