I think it's important to clarify some core points about our system and how it works. Adin's done a good job responding to some of the posts on Slashdot, but we have our hands full cutting the Copper 1.1 release and I figured it would be best to provide a quick overview of what we are up to here, so we can all get back to work :).
The memory oversubscription is one neat application of our core technology, which basically amounts to COW-based cloning of entire virtual machines. Our goal with Copper is to use this primitive to enable whole-cluster virtualization. We take a single physical cluster, and transparently run multiple virtual clusters on top of it. Each of these virtual clusters can grow and shrink independently (by cloning new VMs and killing off clones), to accomodate different demands.
Our ultimate aim is to take 'Virtual Machine Appliances' and turn them into 'Virtual Cluster Appliances': minimal-configuration disk images that are able to spin up a virtual cluster on demand, serve up a single app, and dynamically adjust resource usage based on whatever internal metrics they want to use.
To this end, we've exposed the cloning primitive inside the master virtual machine of Copper virtual clusters. The master VM, through either an API call or a shell tool (which basically wraps the API call), can invoke a 'clone' operation. For example, with the shell, it would look like this:
'gc clone 3'
This will request resources for 3 clone VMs from the Copper controller, create those clones, and then exit. The clones are exact duplicates of the master VM in terms of the state of memory and the root disk. Peripheral devices, external storage resources, and network configurations of the master can also be transparently replicated on the clones.
The semantics of the clone operation closely match those of the familiar UNIX process fork().
There are several additional API calls that can help with managing the resulting virtual cluster - e.g. API calls for listing all clones, killing individual clones or entire generations of clones, etc. Bindings have been created to expose these APIs in several languages so applications can make full use of Copper's capabilities. For instance, David had a very informative post on how to use Copper's APIs to create a dynamic build system (Elastic Build System in Action).
Copper's use of lightweight virtual machine cloning is deceptively powerful for enabling 'virtual cluster appliances'. For example, we can create a 'memcached' virtual cluster that just has memcached installed on the root disk, and can be instantly expanded or contracted in a matter of seconds, with no extra configuration steps. Same can be done for apps like Hadoop and Cassandra. Check out some of the earlier blog posts (Howto: Build and Scale a Cassandra Cluster,Howto: Build a Hadoop Cluster in 5 Minutes , Howto: Build a 10 Node Memcached Cluster) for enabling cluster appliances with Copper.
Virtualization is growing up, and it can't be just about single machines anymore. Copper's name was chosen to be reminiscent of 'cluster operating system', and that's really what we're trying to build at GridCentric.
Copper's use of lightweight virtual machine cloning is deceptively powerful for enabling 'virtual cluster appliances'. For example, we can create a 'memcached' virtual cluster that just has memcached installed on the root disk, and can be instantly expanded or contracted in a matter of seconds, with no extra configuration steps. Same can be done for apps like Hadoop and Cassandra. Check out some of the earlier blog posts (Howto: Build and Scale a Cassandra Cluster,Howto: Build a Hadoop Cluster in 5 Minutes , Howto: Build a 10 Node Memcached Cluster) for enabling cluster appliances with Copper.
Virtualization is growing up, and it can't be just about single machines anymore. Copper's name was chosen to be reminiscent of 'cluster operating system', and that's really what we're trying to build at GridCentric.
0 comments:
Post a Comment