23-05-2014, 04:52 PM
Scalable and Cost-Effective Interconnection of Data-Center Servers Using Dual Server Ports
Scalable and Cost-Effective Interconnection.pdf (Size: 1.27 MB / Downloads: 21)
Abstract
The goal of data-center networking is to interconnect
a large number of server machines with low equipment cost while
providing high network capacity and high bisection width. It
is well understood that the current practice where servers are
connected by a tree hierarchy of network switches cannot meet
these requirements. In this paper, we explore a new server-in-
terconnection structure. We observe that the commodity server
machines used in today’s data centers usually come with two
built-in Ethernet ports, one for network connection and the other
left for backup purposes. We believe that if both ports are actively
used in network connections, we can build a scalable, cost-effective
interconnection structure without either the expensive higher-level
large switches or any additional hardware on servers. We design
such a networking structure called FiConn. Although the server
node degree is only 2 in this structure, we have proven that FiConn
is highly scalable to encompass hundreds of thousands of servers
with low diameter and high bisection width. We have developed
a low-overhead traffic-aware routing mechanism to improve
effective link utilization based on dynamic traffic state. We have
also proposed how to incrementally deploy FiConn.
INTRODUCTION
ATA-CENTER networking designs both the network
structure and associated protocols to interconnect thou-
sands of [8] or even hundreds of thousands of servers [1]–[3] at
a data center, with low equipment cost, high and balanced net-
work capacity, and robustness to link/sever faults. Its operation
is essential to offering both numerous online applications, e.g.,
search, gaming, Web mail, and infrastructure services, e.g.,
GFS [5], Map-reduce [6], and Dryad [7]. It is well understood
that tree-base solution in current practice cannot meet the
requirements [8], [9].
RELATED WORK
Interconnection Structure for Data Centers
We now discuss four interconnection structures proposed for
data centers, the current practice of the tree-based structure, and
two recent proposals of Fat-Tree [8], DCell [9], and BCube [10].
Tree: In current practice, servers are connected by a tree
hierarchy of network switches, with commodity switches at the
first level and increasingly larger and more expensive switches
at the higher levels. It is well known that this kind of tree struc-
ture has many limitations [8], [9]. The top-level switches are
the bandwidth bottleneck, and high-end high-speed switches
have to be used. Moreover, a high-level switch shows as a
single-point failure spot for its subtree branch. Using redundant
switches does not fundamentally solve the problem, but incurs
even higher cost.
Interconnection Rule
FiConn is a recursively defined structure. A high-level
FiConn is constructed by many low-level FiConns. We denote
a level- FiConn as FiConn . FiConn is the basic construction
unit, which is composed of servers and an -port commodity
switch connecting the servers. Typically, is an even number
such as 16, 32, or 48. Every server in FiConn has one port
connected to the switch in FiConn , and we call this port
level-0 port. The link connecting a level-0 port and the switch is
called level-0 link. Level-0 port can be regarded as the original
operation port on servers in current practice. If the backup port
of a server is not connected to another server, we call it an
available backup port. For instance, there are initially servers
each with an available backup port in a FiConn .
INCREMENTAL DEPLOYMENT OF FICONN
In practice, it is much likely that the total number of servers
we need in FiConn does not exactly meet the number of servers
in a certain FiConn . Instead, the number is between FiConn
and FiConn
. We call such a structure an incomplete FiConn.
For incremental deployment, we need to address the intercon-
nection in incomplete FiConns. Our principle is that the inter-
connection should not only retain high bisection width in in-
complete FiConn, but also incur low rewiring cost.
In the construction of complete FiConn, we use a bottom-up
approach as in Algorithm 1. In this way, we first deploy a
complete FiConn , then a complete FiConn , and so forth. One
problem of this approach is that it may generate incomplete
FiConn with low bisection width. For example, if there are
’s in an incomplete FiConn , the two
only two FiConn
’s will be connected by a single level- link. The
FiConn
bisection width of this structure becomes 1, and the single
level- link is the communication bottleneck in the structure.
CONCLUSION
In this paper, we propose FiConn, a novel server-intercon-
nection network structure that utilizes the dual-port configura-
tion existing in most commodity data-center server machines.
It is a highly scalable structure because the total number of
servers it can support is not limited by the number of server
ports or switch ports. It is cost-effective because it requires less
number of switches and links than other recently proposed struc-
tures for data centers. We have designed traffic-aware routing
in FiConn to make better utilization of the link capacities ac-
cording to traffic states. We also have proposed solutions to in-
crease the bisection width in incomplete FiConns during incre-
mental deployment.