27-08-2012, 09:57 AM
Going Back and Forth: Efficient Multideployment and Multisnapshotting on Clouds
Going Back and Forth.pdf (Size: 284.88 KB / Downloads: 33)
ABSTRACT
Infrastructure as a Service (IaaS) cloud computing has rev-
olutionized the way we think of acquiring resources by in-
troducing a simple change: allowing users to lease compu-
tational resources from the cloud provider’s datacenter for a
short time by deploying virtual machines (VMs) on these re-
sources. This new model raises new challenges in the design
and development of IaaS middleware. One of those chal-
lenges is the need to deploy a large number (hundreds or
even thousands) of VM instances simultaneously. Once the
VM instances are deployed, another challenge is to simulta-
neously take a snapshot of many images and transfer them
to persistent storage to support management tasks, such as
suspend-resume and migration. With datacenters growing
rapidly and configurations becoming heterogeneous, it is im-
portant to enable efficient concurrent deployment and snap-
shotting that are at the same time hypervisor independent
and ensure a maximum compatibility with different configu-
rations. This paper addresses these challenges by proposing
a virtual file system specifically optimized for virtual ma-
chine image storage. It is based on a lazy transfer scheme
coupled with object versioning that handles snapshotting
transparently in a hypervisor-independent fashion, ensuring
high portability for different configurations.
INTRODUCTION
In recent years, Infrastructure as a Service (IaaS) cloud
computing [30] has emerged as a viable alternative to the
acquisition and management of physical resources. With
IaaS, users can lease storage and computation time from
large datacenters. Leasing of computation time is accom-
plished by allowing users to deploy virtual machines (VMs)
on the datacenter’s resources. Since the user has complete
control over the configuration of the VMs using on-demand
deployments [5, 16], IaaS leasing is equivalent to purchasing
dedicated hardware but without the long-term commitment
and cost. The on-demand nature of IaaS is critical to mak-
ing such leases attractive, since it enables users to expand
or shrink their resources according to their computational
needs, by using external resources to complement their local
resource base [20].
This emerging model leads to new challenges relating to
the design and development of IaaS systems. One of the
commonly occurring patterns in the operation of IaaS is the
need to deploy a large number of VMs on many nodes of a
datacenter at the same time, starting from a set of VM im-
ages previously stored in a persistent fashion. For example,
this pattern occurs when the user wants to deploy a virtual
cluster that executes a distributed application or a set of en-
vironments to support a workflow. We refer to this pattern
as multideployment.
Application state
The state of the VM deployment is defined at each mo-
ment in time by two main components: the state of each
of the VM instances and the state of the communication
channels between them (opened sockets, in-transit network
packets, virtual topology, etc.).
Thus, in the most general case (Model 1 ), saving the appli-
cation state implies saving both the state of all VM instances
and the state of all active communication channels among
them. While several methods have been established in the
virtualization community to capture the state of a running
VM (CPU registers, RAM, state of devices, etc.), the issue
of capturing the global state of the communication channels
is difficult and still an open problem [19].
In order to avoid this issue, the general case is usually sim-
plified such that the application state is reduced to the sum
of states of the VM instances (Model 2 ). Any in-transit
network traffic is discarded, under the assumption that a
fault-tolerant networking protocol is used that is able to re-
store communication channels and resend lost information.
Even so, for VM instances that need large amounts of
memory, the necessary storage space can explode to huge
sizes. For example, saving 2 GB of RAM for 1,000 VMs con-
sumes 2 TB of space, which is unacceptable for a single one-
point-in-time deployment checkpoint.
Applicability in the cloud
The simplified architecture of a cloud that integrates our
approach is depicted in Figure 1. The typical elements found
in the cloud are illustrated with a light background, while
the elements that are part of our proposal are highlighted by
a darker background. A distributed versioning storage ser-
vice that supports cloning and shadowing is deployed on the
compute nodes and consolidates parts of their local disks
into a common storage pool. The cloud client has direct
access to the storage service and is allowed to upload and
download images from it. Every uploaded image is automat-
ically striped. Furthermore, the cloud client interacts with
the cloud middleware through a control API that enables a
variety of management tasks, including deploying an image
on a set of compute nodes, dynamically adding or removing
compute nodes from that set, and snapshotting individual
VM instances or the whole set. The cloud middleware in
turn coordinates the compute nodes to achieve the afore-
mentioned management tasks. Each compute node runs a
hypervisor that is responsible for running the VMs. The
reads and writes of the hypervisor are trapped by the mir-
roring module, which is responsible for on-demand mirroring
and snapshotting (as explained in Section 3.1) and relies on
both the local disk and the distributed versioning storage
service to do so.
IMPLEMENTATION
In Section 3.2 we illustrated how to apply our approach
in the cloud by means of two basic building blocks: a dis-
tributed versioning storage service, which supports cloning
and shadowing and is responsible for managing the reposi-
tory, and a mirroring module, which runs on each compute
node and is responsible for trapping the I/O accesses of the
hypervisor to the image with the purpose of facilitating on-
demand mirroring and snapshotting.
In this section we show how to efficiently implement these
building blocks in such a way that they achieve the design
principles introduced in Section 3.1 on the one hand and are
easy to integrate in the cloud on the other hand.
Dependencies
We have implemented the distributed versioning storage
service on top of BlobSeer [23, 24, 25]. This choice was mo-
tivated by several factors. First, BlobSeer enables scalable
aggregation of storage space from the participating nodes
with minimal overhead in order to store BLOBs (Binary
Large OBjects). Data striping is performed transparently on
BLOBs, which enables direct mapping between BLOBs and
virtual machine images and therefore eliminates the need for
explicit chunk management. Second, BlobSeer offers out-of-
the-box support for shadowing, which significantly simpli-
fies the implementation of the COMMIT primitive. Third,
the service was optimized to sustain a high throughput even
under heavy access concurrency, which is especially useful in
the context of the multideployment and multisnapshotting
patterns because it enables efficient parallel access to the
image chunks.
RELATEDWORK
Multideployment that relies on full broadcast-based pre-
propagation is a widely used technique [28, 31, 14]. While
this technique avoids read contention to the repository, it
can incur a high overhead in both network traffic and ex-
ecution time, as presented in Section 5.2. Furthermore,
since the VM images are fully copied locally on the compute
nodes, multisnapshotting becomes infeasible: large amounts
of data are unnecessarily duplicated and cause unacceptable
transfer delays, not to mention huge storage space and net-
work traffic utilization.
CONCLUSIONS
As cloud computing becomes increasingly popular, effi-
cient management of VM images, such as image propaga-
tion to compute nodes and image snapshotting for check-
pointing or migration, is critical. The performance of these
operations directly affects the usability of the benefits of-
fered by cloud computing systems. This paper introduced
several techniques that integrate with cloud middleware to
efficiently handle two patterns: multideployment and multi-
snapshotting.
Going Back and Forth.pdf (Size: 284.88 KB / Downloads: 33)
ABSTRACT
Infrastructure as a Service (IaaS) cloud computing has rev-
olutionized the way we think of acquiring resources by in-
troducing a simple change: allowing users to lease compu-
tational resources from the cloud provider’s datacenter for a
short time by deploying virtual machines (VMs) on these re-
sources. This new model raises new challenges in the design
and development of IaaS middleware. One of those chal-
lenges is the need to deploy a large number (hundreds or
even thousands) of VM instances simultaneously. Once the
VM instances are deployed, another challenge is to simulta-
neously take a snapshot of many images and transfer them
to persistent storage to support management tasks, such as
suspend-resume and migration. With datacenters growing
rapidly and configurations becoming heterogeneous, it is im-
portant to enable efficient concurrent deployment and snap-
shotting that are at the same time hypervisor independent
and ensure a maximum compatibility with different configu-
rations. This paper addresses these challenges by proposing
a virtual file system specifically optimized for virtual ma-
chine image storage. It is based on a lazy transfer scheme
coupled with object versioning that handles snapshotting
transparently in a hypervisor-independent fashion, ensuring
high portability for different configurations.
INTRODUCTION
In recent years, Infrastructure as a Service (IaaS) cloud
computing [30] has emerged as a viable alternative to the
acquisition and management of physical resources. With
IaaS, users can lease storage and computation time from
large datacenters. Leasing of computation time is accom-
plished by allowing users to deploy virtual machines (VMs)
on the datacenter’s resources. Since the user has complete
control over the configuration of the VMs using on-demand
deployments [5, 16], IaaS leasing is equivalent to purchasing
dedicated hardware but without the long-term commitment
and cost. The on-demand nature of IaaS is critical to mak-
ing such leases attractive, since it enables users to expand
or shrink their resources according to their computational
needs, by using external resources to complement their local
resource base [20].
This emerging model leads to new challenges relating to
the design and development of IaaS systems. One of the
commonly occurring patterns in the operation of IaaS is the
need to deploy a large number of VMs on many nodes of a
datacenter at the same time, starting from a set of VM im-
ages previously stored in a persistent fashion. For example,
this pattern occurs when the user wants to deploy a virtual
cluster that executes a distributed application or a set of en-
vironments to support a workflow. We refer to this pattern
as multideployment.
Application state
The state of the VM deployment is defined at each mo-
ment in time by two main components: the state of each
of the VM instances and the state of the communication
channels between them (opened sockets, in-transit network
packets, virtual topology, etc.).
Thus, in the most general case (Model 1 ), saving the appli-
cation state implies saving both the state of all VM instances
and the state of all active communication channels among
them. While several methods have been established in the
virtualization community to capture the state of a running
VM (CPU registers, RAM, state of devices, etc.), the issue
of capturing the global state of the communication channels
is difficult and still an open problem [19].
In order to avoid this issue, the general case is usually sim-
plified such that the application state is reduced to the sum
of states of the VM instances (Model 2 ). Any in-transit
network traffic is discarded, under the assumption that a
fault-tolerant networking protocol is used that is able to re-
store communication channels and resend lost information.
Even so, for VM instances that need large amounts of
memory, the necessary storage space can explode to huge
sizes. For example, saving 2 GB of RAM for 1,000 VMs con-
sumes 2 TB of space, which is unacceptable for a single one-
point-in-time deployment checkpoint.
Applicability in the cloud
The simplified architecture of a cloud that integrates our
approach is depicted in Figure 1. The typical elements found
in the cloud are illustrated with a light background, while
the elements that are part of our proposal are highlighted by
a darker background. A distributed versioning storage ser-
vice that supports cloning and shadowing is deployed on the
compute nodes and consolidates parts of their local disks
into a common storage pool. The cloud client has direct
access to the storage service and is allowed to upload and
download images from it. Every uploaded image is automat-
ically striped. Furthermore, the cloud client interacts with
the cloud middleware through a control API that enables a
variety of management tasks, including deploying an image
on a set of compute nodes, dynamically adding or removing
compute nodes from that set, and snapshotting individual
VM instances or the whole set. The cloud middleware in
turn coordinates the compute nodes to achieve the afore-
mentioned management tasks. Each compute node runs a
hypervisor that is responsible for running the VMs. The
reads and writes of the hypervisor are trapped by the mir-
roring module, which is responsible for on-demand mirroring
and snapshotting (as explained in Section 3.1) and relies on
both the local disk and the distributed versioning storage
service to do so.
IMPLEMENTATION
In Section 3.2 we illustrated how to apply our approach
in the cloud by means of two basic building blocks: a dis-
tributed versioning storage service, which supports cloning
and shadowing and is responsible for managing the reposi-
tory, and a mirroring module, which runs on each compute
node and is responsible for trapping the I/O accesses of the
hypervisor to the image with the purpose of facilitating on-
demand mirroring and snapshotting.
In this section we show how to efficiently implement these
building blocks in such a way that they achieve the design
principles introduced in Section 3.1 on the one hand and are
easy to integrate in the cloud on the other hand.
Dependencies
We have implemented the distributed versioning storage
service on top of BlobSeer [23, 24, 25]. This choice was mo-
tivated by several factors. First, BlobSeer enables scalable
aggregation of storage space from the participating nodes
with minimal overhead in order to store BLOBs (Binary
Large OBjects). Data striping is performed transparently on
BLOBs, which enables direct mapping between BLOBs and
virtual machine images and therefore eliminates the need for
explicit chunk management. Second, BlobSeer offers out-of-
the-box support for shadowing, which significantly simpli-
fies the implementation of the COMMIT primitive. Third,
the service was optimized to sustain a high throughput even
under heavy access concurrency, which is especially useful in
the context of the multideployment and multisnapshotting
patterns because it enables efficient parallel access to the
image chunks.
RELATEDWORK
Multideployment that relies on full broadcast-based pre-
propagation is a widely used technique [28, 31, 14]. While
this technique avoids read contention to the repository, it
can incur a high overhead in both network traffic and ex-
ecution time, as presented in Section 5.2. Furthermore,
since the VM images are fully copied locally on the compute
nodes, multisnapshotting becomes infeasible: large amounts
of data are unnecessarily duplicated and cause unacceptable
transfer delays, not to mention huge storage space and net-
work traffic utilization.
CONCLUSIONS
As cloud computing becomes increasingly popular, effi-
cient management of VM images, such as image propaga-
tion to compute nodes and image snapshotting for check-
pointing or migration, is critical. The performance of these
operations directly affects the usability of the benefits of-
fered by cloud computing systems. This paper introduced
several techniques that integrate with cloud middleware to
efficiently handle two patterns: multideployment and multi-
snapshotting.