Towards Seamless Live
Migration in SDN-Based
Data Centers
Kyoomars Alizadeh Noghani
K
yoomars Alizadeh N
oghani | T
owards Seamless L
ive Migration in S
D
N-Based Data Centers |
2018:55
Towards Seamless Live Migration in
SDN-Based Data Centers
Live migration of Virtual Machines (VMs) has significantly improved the
flexibility of modern Data Centers (DCs). Ideally, live migration ought to be
seamless which requires a comprehensive support from the underlying network.
However, legacy DC networks fall short to address the challenges of migration
due to their inflexible and decentralized characteristics. In contrast, Software
Defined Networking (SDN) is a new networking paradigm, which has the
potential to improve the live migration thanks to its comprehensive view over
the network, flexible structure, and its close integration with DC management
infrastructures.
This thesis investigates networking challenges of short and long-haul live VM
migration in SDN-based DCs. We propose solutions to make the intra- and
inter-DC live migration procedures more seamless. Furthermore, our proposed
SDN-based framework for inter-DC migration improves the management, enhances
the performance, and increases the scalability of interconnections among DCs.
LICENTIATE THESIS | Karlstad University Studies | 2018:55
Faculty of Health, Science and Technology
Computer Science
LICENTIATE THESIS | Karlstad University Studies | 2018:55
ISSN 1403-8099
ISBN 978-91-7063-991-3 (pdf)
ISBN 978-91-7063-896-1 (print)
LICENTIATE THESIS | Karlstad University Studies | 2018:55
Towards Seamless Live
Migration in SDN-Based
Data Centers
Print: Universitetstryckeriet, Karlstad 2018
Distribution:
Karlstad University
Faculty of Health, Science and Technology
Department of Mathematics and Computer Science
SE-651 88 Karlstad, Sweden
+46 54 700 10 00
©
The author
ISSN 1403-8099
urn:nbn:se:kau:diva-70166
Karlstad University Studies | 2018:55
LICENTIATE THESIS
Kyoomars Alizadeh Noghani
Towards Seamless Live Migration in SDN-Based Data Centers
WWW.KAU.SE
ISBN 978-91-7063-991-3 (pdf)
Towards Seamless Live Migration in SDN-Based
Data Centers
Kyoomars Alizadeh Noghani
Department of Computer Science, Karlstad University, Sweden
Abstract
Live migration of Virtual Machines (VMs) has significantly improved the flex-ibility of modern Data Centers (DCs). Ideally, live migration ought to be seamless which in turn raises challenges on how to minimize service disruption and avoid performance degradation. To address these challenges, a compre-hensive support from the underlying network is required. However, legacy DC networks fall short to help as they take a reactive approach to live migration procedure. Moreover, the complexity and inflexibility of legacy DC networks make it difficult to deploy, manage, and improve network technologies that DC providers may need to use for migration.
In this thesis, we explore the application of Software Defined Network-ing (SDN) paradigm for makNetwork-ing live VM migration more seamless. Exploit-ing the characteristics of SDN such as its centralized view on network states, we contribute to the body of knowledge by enhancing the quality of intra-and inter-DC live migration. Firstly, for intra-DC migration, we provide an SDN-based solution which minimizes the service disruption by employing OpenFlow-based resiliency mechanisms to prepare a DC network for migra-tion proactively. Secondly, we improve the inter-DC live migramigra-tion by acceler-ating the network convergence through announcing the migration in the con-trol plane using MP-BGP protocol. Further, our proposed framework resolves the sub-optimal routing problem by conducting the gateway functionality at the SDN controller. Finally, with the ultimate goal of improving the inter-DC migration, we develop an SDN-based framework which automates the deploy-ment, improves the managedeploy-ment, enhances the performance, and increases the scalability of interconnections among DCs.
Keywords: Data Center, Data Center Interconnection, EVPN, SDN, VM Migration.
v
Acknowledgments
PhD students, to be successful, should eat the PhD elephant one bite at a time. Quite sure that my licentiate thesis is not just half of the elephant as I am preparing to defend my final PhD thesis approximately in a year. Howev-er, it does not diminish the benefits I will receive from presenting my work at this current important stage. Not only I may receive valuable feedback, but also the acknowledgement page provides me the opportunity to appreciate the people who helped me to have a better personal and professional life.
First and foremost, I would like to express my endless gratitude to my main supervisor, Professor Andreas Kassler for his insightful advice, reliable guidance, and full support. Next, I would like to express my sincere thanks to my committed and punctual co-supervisor, Associate Professor Karl-Johan Grinnemo, who cared so much about my work. I am grateful to Professor Andrei Gurtov for reviewing my Licentiate proposal and accepting the role of opponent in my Licentiate thesis defense. I would also like to thank all my co-authors, colleagues, and friends from the Department of Computer Science at Karlstad University.
My deep and sincere gratitude to my parents, brother, sister, and my in-laws for their continuous and unconditional love, encouragement, and sup-port.
I dedicate this milestone to my beloved wife, Farzaneh. Thank you for your love and understanding. I am utterly blessed to have you in my life.
vii
List of Appended Papers
This thesis is based on the work reported in the following appended papers. I. Cristian Hernandez Benet, Kyoomars Alizadeh Noghani, and Andreas
Kassler. Minimizing Live VM Migration Downtime Using OpenFlow based Resiliency Mechanisms. In 5th IEEE Conference on Cloud Network-ing (Cloudnet), Pisa, Italy, October 3–5, 2016.
II. Kyoomars Alizadeh Noghani, Cristian Hernandez Benet, Andreas Kassler, Antonio Marotta, Patrick Jestin, and Vivek Srivastava. Automating Eth-ernet VPN Deployment in SDN-based Data Centers. In 4th IEEE Confer-ence on Software Defined Systems (SDS), Valencia, Spain, May 8–11, 2017. III. Cristian Hernandez Benet, Kyoomars Alizadeh Noghani, Andreas Kassler,
Ognjen Dobrijevic, and Patrick Jestin. Policy-based Routing and Load Balancing for EVPN-based Data Center Interconnections. In IEEE Con-ference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Berlin, Germany, November 6–8, 2017.
IV. Kyoomars Alizadeh Noghani and Andreas Kassler. SDN Enhanced Ethernet VPN for Data Center Interconnect. In 6th IEEE Conference on Cloud Networking (Cloudnet), Prague, Czech Republic, September 25– 27, 2017.
V. Kyoomars Alizadeh Noghani, Andreas Kassler, and Prem Sankar Gopan-nan. EVPN/SDN Assisted Live VM Migration between Geo-Distributed Data Centers. In 4th IEEE Conference on Network Softwarization (Net-Soft), Montreal, Canada, June 25–29, 2018.
Note: Some of the appended papers have been subjected to minor editorial changes.
Comments on my Participation
Paper I The initial idea of the paper originated from discussions with my colleague, Christian Hernandez Benet. I participated in developing all pro-posed resiliency solutions to address the challenges of intra-DC VM migra-tion. Additionally, I was actively involved in writing the paper except for the evaluation section.
Paper II I designed, developed, and implemented the proposed framework as well as conducted the experiments for evaluations. Moreover, I am the principal author of all parts of the paper. My co-authors assisted me in the evaluation section and writing the paper.
Paper III Christian Hernandez Benet, is the main author of this paper. I actively participated in developing the architecture and traffic engineering
viii
policies, as well as writing the initial draft of the paper.
Paper IV I am the main author of the paper. The idea of the paper came from reading the IETF documents about the EVPN technology. I proposed an SDN-based solution and conducted experiments for evaluations.
Paper V The idea of the paper came from watching Cisco summits on data center networks. I further investigated the problem, proposed an SDN-based solution, and carried out all the experimental evaluations presented in the pa-per. Furthermore, I authored all sections of the papa-per.
Other Publications
• Cristian Hernandez Benet, Robayet Nasim, Kyoomars Alizadeh Noghani, and Andreas Kassler. OpenStackEmu - A Cloud Testbed Combining Network Emulation with OpenStack and SDN. In 14th IEEE Annual Consumer Communications Networking Conference (CCNC), Las Vegas, NV, USA, January 8–11, 2017.
• Cristian Hernandez Benet, Kyoomars Alizadeh Noghani, and Javid Taheri. SDN Implementations and Protocols. In Big Data and Software Defined Networks, Chapter 2, Pages 27-48, The Institution of Engineer-ing & Technology, March 2018.
• Kyoomars Alizadeh Noghani, Cristian Hernandez Benet, and Javid Taheri. SDN helps Volume in Big Data. In Big Data and Software De-fined Networks, Chapter 9, Pages 185-205, The Institution of Engineer-ing & Technology, March 2018.
• Abdelmounaam Rezgui, Kyoomars Alizadeh Noghani, Javid Taheri, Amir Mirzaeinia, Hamdy Soliman, and Nikolas Davis. SDN helps Big Data to Become Fault Tolerant. In Big Data and Software Defined Net-works, Chapter 15, Pages 319-336, The Institution of Engineering & Technology, March 2018.
ix
Contents
List of Appended Papers vii
Introductory Summary
1
1 Introduction 3
2 Background 5
2.1 Live VM Migration . . . 5
2.2 SDN-based Resiliency Mechanisms . . . 6
2.3 VXLAN . . . 6
2.4 EVPN . . . 6
2.5 Model Driven Network Management . . . 8
3 Related Work 9 3.1 Live VM Migration . . . 9
3.1.1 Retain Network Connectivity . . . 9
3.1.2 Large Convergence Time . . . 10
3.1.3 Sub-Optimal Routing . . . 10
3.2 EVPN Automation and Management . . . 11
3.3 EVPN Policy . . . 11
3.4 EVPN Scalability . . . 11
4 Research Questions 12 5 Contributions 13 6 Research Methodology 14 7 Summary of Appended Papers 16 8 Conclusions and Future Work 18
Paper
I:
Minimizing Live VM Migration Downtime Using
Open-Flow based Resiliency Mechanisms
27
1 Introduction 29 2 Background 31 2.1 SDN-based Resiliency Mechanisms . . . 31x
3 Flow Restoration for VM Migration 33
3.1 Legacy Network based Live VM Migration . . . 33
3.2 SDN-based Live VM Migration . . . 34
3.3 SDN-based Live VM Migration with FastFailover . . . 34
3.4 SDN-based Live VM Migration with Packet bicasting . . . 35
3.5 SDN-based Live VM Migration using Stateful Forwarding . . 36
4 Experimental Evaluation 37 5 Conclusion 40
Paper
II:
Automating Ethernet VPN Deployment in SDN-based
Da-ta Centers
43
1 Introduction 45 2 Background 47 3 Architecture and Implementation 48 3.1 High-level Architecture . . . 483.2 Enhanced SDN Functionalities for EVPN . . . 49
3.3 SDN Controller Modules . . . 50 3.3.1 Neutron . . . 50 3.3.2 L2VPN Service . . . 50 3.3.3 BGP-EVPN . . . 51 3.3.4 PEConfigure . . . 52 3.3.5 Existing modules . . . 52 4 Evaluation 53 4.1 Evaluation Methodology . . . 53
4.2 EVPN Deployment Performance . . . 53
4.3 Module Performance Test . . . 55 5 Conclusions and Future Work 57
Paper
III:
Policy-based Routing and Load Balancing for EVPN-based
Data Center Interconnections
61
1 Introduction 63
2 Background and Related Work 64
xi
4 Proposed SDN-based Framework 66
4.1 SDN Controller Modules . . . 67
4.1.1 The Neutron module . . . 67
4.1.2 The L2VPN Service module . . . 67
4.1.3 The Policy Manager (PM) module . . . 67
4.1.4 The Strategy Manager module . . . 68
4.2 Routing Policy Attributes . . . 69
4.2.1 Multi-Homing . . . 69
4.2.2 Load Balancing . . . 70
4.2.3 Bandwidth Reservation . . . 70
4.3 Policy Enforcement . . . 70
4.4 Exemplary Work Flow . . . 71
5 Evaluation and Results 72 5.1 Evaluation Methodology . . . 72
5.2 Policies . . . 74
5.2.1 No Multi-Homing (NO_MH) . . . 74
5.2.2 Multi-Homing . . . 74
5.2.3 Multi-Homing and Load Balancing (MHLB) . . . 74
5.2.4 Load Balancing, but Not Multi-Homing (LB_NO_MH) 75 5.2.5 Bandwidth Guarantee QoS (QoS) . . . 75
5.3 Results . . . 75
6 Conclusions and Future Work 78
Paper
IV:
SDN Enhanced Ethernet VPN for Data Center
Intercon-nect
81
1 Introduction 83 2 Background 85 3 Proposed Architecture 86 3.1 BUM Traffic Routing . . . 873.2 Multicast Tree Inside a DC . . . 89
3.3 Proposed Solution . . . 89
3.4 Using SDN Controller for DF Selection . . . 90
4 Evaluation 91 4.1 Experimental Methodology . . . 91
4.2 DF Switch-Over . . . 92
4.3 SDN Controller Triggered DF Change . . . 94
xii
Paper
V:
EVPN/SDN Assisted Live VM Migration between
Geo-Distributed Data Centers
99
1 Introduction 101
2 Design Challenges for live VM migration across the WAN 103
3 Background 105
3.1 VXLAN and EVPN . . . 105
3.2 VM Mobility in EVPN . . . 106
3.3 Distributed Gateway using EVPN . . . 107
4 Architecture and Implementation 108 4.1 Controller Modules . . . 108
4.1.1 L2VPN-Service . . . 108
4.1.2 BGP-EVPN . . . 109
4.1.3 VXLAN-Manager . . . 110
4.2 Improving Network Convergence Time across DCs . . . 110
4.3 Addressing the Hair-Pinning Problem . . . 112
5 Evaluation 113 5.1 Intra Subnet . . . 115
5.2 Inter Subnet . . . 118
Towards Seamless Live Migration in SDN-Based Data Centers 3
1 Introduction
Server virtualization has significantly improved levels of efficiency, agility, availability, and flexibility of modern Data Centers (DCs). The ability to seamlessly migrate a Virtual Machine (VM) between physical servers improves the flexibility, makes DCs more fault-tolerant, and helps DC providers to achieve a wide range of objectives such as dynamic load balancing.
Live VM migration needs to be seamless which requires maintaining ongo-ing connections, providongo-ing a negligible downtime, and avoidongo-ing performance degradation after the migration. To conduct a seamless migration, a compre-hensive support from the underlying network is as crucial as exploiting an optimal migration scheme to transfer the system states. A network can sub-stantially improve the migration procedure in the following ways:
Preserve Network Connectivity: The ability to maintain ongoing con-nections is a prerequisite for seamless VM migration. Typically, migration of a VM between different networks may require the IP address(es) of the VM to be changed. As a result, ongoing connections of the VM ought to be re-established which respectively violates the seamless feature of live VM migra-tion. A network can provide an opportunity for the migrating VM to preserve its network connections, for instance, by using overlay network technologies. Fast Network Convergence: Unless a network is not informed about the new location of the migrated VM, the peers of the VM continue sending traffic to its former location which consequently introduces further service interruption. Therefore, it is important that the network reduces the inter-ruption interval by promptly re-routing the north and south traffic of the VM to its new location.
Optimal Traffic Routing: When a VM migrates, the network ought to find a new routing path for its ingress and egress traffic in accordance with its new location. Using a sub-optimal path for the ingress and egress traffic of a migrating VM may significantly degrade the application performance and introduce problems such as congestion in the network.
Performance Improvement of Migration: A network can significantly reduce the migration downtime by selecting an appropriate path to transfer the system states of a VM. Moreover, the network can utilize various technolo-gies and protocols to improve the performance of live migration. For instance, the network can prioritize the migration traffic using protocols such as Differ-entiated Services (DiffServ) [13]. Additionally, if the network is intrinsically designed with a high degree of available path diversity, then new transmission protocols such as Multipath TCP (MPTCP) [23] can substantially speed up the migration procedure [40].
Performance Improvement of Required Technologies: To conduct a seamless migration, network providers may need to employ various technolo-gies. For instance, DC providers interconnect their remote sites through over-lay networks or Layer 2 Virtual Private Networks (L2VPNs) solutions when they want to conduct inter-DC migration. Typically, these technologies are
4 Introductory Summary
naturally decentralized as they are developed to operate in legacy networks. Depending on the network size, reaching a convergence in decentralized sys-tems is time-consuming. Moreover, the output of the convergence can be sub-optimal as each entity makes a decision according to its local view at a given time. The migration will improve if a network addresses the problems of the involved technologies (e.g., scalability problems).
However, legacy DC networks fall short to provide the aforementioned functionalities due to their inflexible and decentralized characteristics, as well as reactive approaches they take to live migration. Furthermore, conducting a seamless VM migration in legacy DC networks is difficult as it requires a no-table amount of effort to configure the underlying network, for instance, to provide interconnection among DCs. Software Defined Networking (SDN) has revolutionized the traditional network architecture, allowing for new ways to address the network constraints. In the context of live VM migration, SDN can help the procedure in the following ways:
1) The holistic view of the SDN controller over the network provides oppor-tunities to i) determine optimal routing paths for migration traffic, ii) manage the ingress and egress traffic of the migrating VM efficiently, iii) prepare the network for migration, for instance, by installing new forwarding rules while hypervisor transfers the system states of the VM, and iv) propagate appropri-ate messages in the control plane to advertise the migration when it is required. 2) The SDN controller may benefit from a tight integration with public cloud platforms such as OpenStack [7] to automatically deploy and flexibly manage various technologies which may be used for VM migration.
3) The SDN controller can speed up the convergence procedure, improve per-formance, and enhance the scalability of decentralized network technologies which may be used for VM migration.
The ultimate goal of this thesis is to improve the live VM migration pro-cedure by addressing related networking challenges using SDN-based archi-tecture. Specifically, this thesis targets two problems that legacy networks are unable to tackle as well as those of other problems that affect technologies required to conduct a seamless migration. Slow network convergence and sub-optimal routing problem are two network challenges that are addressed here-in. In the context of intra-DC migration, a number of SDN-based resiliency mechanisms are proposed to decrease the network convergence time. Later, the controller re-optimizes the path to remove sub-optimal routing problems. For inter-DC migration, a novel SDN-based approach is presented that accel-erates the network convergence through message passing in the control plane and optimizes the post-migration traffic routing.
Additionally, this thesis attempts to improve the Ethernet Virtual Private Network (EVPN) technology [20]. EVPN, which is an L2 interconnection solution, is selected to be improved as it plays a key role in inter-DC migration scenarios. EVPN helps the VM to retain its ongoing connections while it migrates between remote sites. Moreover, EVPN is designed to address the requirements of modern DCs such as VM mobility and fine-grained traffic
Towards Seamless Live Migration in SDN-Based Data Centers 5
load balancing. In this thesis, an SDN-based framework is developed inside the OpenDaylight (ODL) [6] controller to automate the deployment and improve the management of EVPN-based interconnections. The aforemen-tioned framework is further extended to improve the performance and scal-ability of such interconnections by deploying routing policies and handling the broadcast traffic in a better way.
The remainder of this thesis is organized as follows. Section 2 provides an overview of the technologies that are used through this thesis. Related works are discussed in Section 3. The research questions and contributions are out-lined in Sections 4 and 5, respectively. The research methods employed by the appended papers are discussed in Section 6. Section 7 provides a summa-ry of all the appended papers. Finally, Section 8 concludes the introductosumma-ry summary of this work.
2 Background
This section provides an overview of the underlying concepts and technologies that are used to improve live VM migration in this thesis. The discussion starts with an overview of live VM migration. Then the discussion proceeds with a description of SDN-based resiliency mechanism as it is the main concept used to address the challenges of intra-DC live migration. Finally, this section concisely explains Virtual Extensible LAN (VXLAN), EVPN, and model-driven network management technologies. The aforementioned technologies are exploited in this thesis to improve the live migration procedure.
2.1 Live VM Migration
In live VM migration, a VM transfers its states such as CPU, associated mem-ory, and storage from one physical server to another. There are mostly three different schemes for live VM migration: pre-copy [18], post-copy [27], and hybrid [46]. All these migration schemes are constituted of the following phases: i) initialization, ii) reservation, iii) iteration, iv) stop-and-copy, v) commitment, and vi) activation. Initialization and reservation are conducted before the VM states are transferred to the destination. In the initialization, the host is checked for compatibility of images, CPU architecture, etc. During the reservation, resources on the destination host required for the new VM are reserved. In the iteration phase, the system states of the VM are transferred from the source to the destination node over several iterations while the VM is still providing its services. In the stop-and-copy phase, the VM stops servicing clients at the source node and transfers its latest system states including the modified or remained memory pages to the destination node. The last two steps are performed after the stop-and-copy phase is finished. In the commit-ment phase, the destination host acknowledges receiving the consistent copy of the VM and finally the VM starts after the activation phase [9].
Besides the transmission of VM, its corresponding traffic has also to be resumed to finish the migration procedure. To do so, once network devices
6 Introductory Summary
are informed about the new location of the VM, they update their routing in-formation and steer the north and south traffic of the VM to its new location. In this thesis, we propose to update routing tables during the initialization (Paper I) and stop-and-copy phase (Paper V) for intra- and inter-DC migra-tion, respectively. The rationale behind this idea is to reduce the migration downtime by conducting independent tasks in parallel.
2.2 SDN-based Resiliency Mechanisms
Typically, SDN mechanisms to cope with network failures are classified into two general approaches: i) recovery, and ii) protection. The recovery scheme requires communication between a switch and its controller in order to dy-namically provide backup paths. Once a link/node fails, the controller has to be notified which then reacts by finding an alternative path. Depending on the workload of the controller this procedure may require a significant amount of time. In contrast, in the protection schemes the network is designed in advance to cope with failures. In OpenFlow-enabled networks, protection schemes are typically implemented using OpenFlow group tables. A flow rule, in an OpenFlow group table, can be defined based on several action buckets in which actions are defined based on status parameters. A predefined action is then executed locally without involvement of the controller once param-eters change. To prepare the network for intra-DC VM migration, several OpenFlow-based protection mechanisms have been exploited in Paper I.
2.3 VXLAN
To conduct a seamless migration, the network ought to prevent the ongoing connections of the migrating VM from being re-established. To do so, the network can either provide an opportunity for the migrating VM to maintain its configuration (e.g., VLAN) or convert the old configuration to the new one by manipulating the south and north traffic of the VM. However, not all solutions can be deployed in modern DCs as they are not designed for multi-tenant environments and have major scalability problems. VXLAN [35] is an overlay technology that provides L2 extension over a shared L3 underlay infrastructure network by using MAC in IP/UDP tunneling encapsulation. In the VXLAN-based network, a VM can retain its network configuration while the multi-tenancy requirements are provided at scale and the tenant’s traffic are clearly isolated. In this thesis we assume that the VXLAN overlay technology is deployed inside all DCs. As a result, a VM can retain its network configuration while it migrates inside a DC network.
2.4 EVPN
Although overlay technologies such as VXLAN are widely deployed in DC networks, they are not designed to be a DC interconnect solution. Extend-ing the overlay network across DCs expands the broadcast domain from one
Towards Seamless Live Migration in SDN-Based Data Centers 7
DC network to another which consequently introduces scalability, efficiency, and security problems. Instead, L2VPN is a common solution that network providers use to stretch the layer 2 domain between their remote sites.
EVPN encompasses the next-generation Ethernet L2VPN solutions and has been designed to provide per-flow load balancing, enhance the flexibili-ty, improve the scalabiliflexibili-ty, and decrease the operational complexity of exist-ing L2VPN solutions. EVPN aligns the well-understood technical and oper-ational principles of IP VPNs to Ethernet services by utilizing MP-BGP in the control plane as the signaling method which removes the need for tradi-tional flood-and-learn1in the data plane. EVPN in conjunction with VXLAN
overlay technology is an appropriate solution to span layer 2 domains between multiple DCs [44].
EVPN comprises four types of messages: Ethernet auto-discovery, Ether-net segment, inclusive multicast, and MAC/IP advertisement route. In the following, we briefly describe the MAC advertisement message and its cor-responding extended community as they have been used in this thesis. For the description and use cases for other routing messages, we refer the reader to [20].
MAC Advertisement: The EVPN MAC/IP advertisement message is de-signed to advertise MAC/IP reachability information of a given VM. When an EVPN capable node is informed about a new MAC address, it advertises the information to its peers through the MP-BGP protocol. All remote peers that belong to the same EVPN instance import this route and insert the an-nounced MAC address and its reachability information (e.g., Ethernet tag2)
into their MAC VRF (Virtual Routing and Forwarding) table. This process allows the remote nodes to know where to send the traffic [20].
VM Mobility: By adding an additional extended community section to the MAC/IP advertisement message, EVPN capable nodes can update each other about VM movement. Every MAC mobility event for a given MAC ad-dress contains a sequence number that increases with each MAC move. This is used by EVPN capable nodes to ensure that the MAC advertisements are pro-cessed correctly. An EVPN capable node advertises a MAC address for the first time with no MAC mobility extended community attribute. When an-other EVPN capable node detects a locally attached MAC address for which it had previously received a MAC/IP advertisement route, it advertises the MAC address in a MAC/IP advertisement route. The advertisement route is tagged with a MAC mobility extended community attribute with a sequence number one greater than the last received sequence number [20]. Fig. 1 illus-trates an EVPN operational scenario. A PE (PE-1) advertises a newly learned MAC address provisioned on a customer network (DC-1) to its peers (PE-2 and PE-3) with no additional extended community attribute (Fig.1a). Later, the VM migrates between remote sites. As it is shown in Fig. 1b, the PE of the DC on the right side (PE-3) re-advertises the MAC address of the
migrat-1In the context of L2VPNs, the flood-and-learn is the procedure of disseminating mac-address information in the dataplane for the remote PE to learn.
8 Introductory Summary
ed VM with an updated sequence number in conjunction with some other updated parameters in the MAC advertisement message.
(a) Initial advertisement
(b) Advertisement after the first migration
Figure 1: EVPN MAC mobility scenario.
Distributed Gateway: EVPN offers a unique and scalable solution which allows gateways to be actively distributed across an arbitrary number of net-work elements. This is especially relevant in cloud environments where a ten-ant may exist or migrate anywhere in the network. Using the combination of MAC/IP advertisement message and default gateway extended community, an EVPN capable node can distribute the gateway information to its peers. The remote peers treat the received MAC/IP address equivalent to their own gateway interface for the purposes of gateway processing. As a result, the gate-way is distributed around all DC networks that are part of the same EVPN instance.
In Paper V, EVPN is the key technology that is used for inter-DC VM migration. First, it is used as the solution that interconnects remote sites. Sec-ond, its capability in advertising the migration is improved in a way to decrease the network convergence time. Finally, the EVPN capability to distribute the gateway information is used to resolve the sub-optimal routing problem.
2.5 Model Driven Network Management
By increasing the size of networks and emerging DCs, it is getting more difficult for infrastructure providers to configure the network devices. Large networks
Towards Seamless Live Migration in SDN-Based Data Centers 9
are usually multi-vendor where each network element is configured in a dif-ferent way, e.g., using difdif-ferent command line interface. As a result, there is a clear need across the industry to simplify the configuration and management for both networks and devices. Model-driven network management auto-mates and accelerates the procedure of creating services through the whole network. In model-driven network management, a data model is used for representing services and configurations together with standard protocols to transmit the modeled data. YANG [12] has clearly positioned itself as the da-ta model language for representing network device configurations, sda-tate dada-ta, remote procedure calls, and process notifications in a standard way. Data de-fined in YANG is transmitted to a network device using a protocol such as NETCONF [22]. Over the last couple of years, YANG and NETCONF have gained traction in the networking industry and there is a growing set of products from all vendors supporting YANG as data model and NETCONF as the network management protocol. In Paper II, the SDN controller uses the model-driven network management to automate the configuration of EVPN instances on edge routers of a DC.
3 Related Work
This section describes the research related to the work presented in this thesis.
3.1 Live VM Migration
In general, there are two ways to improve the performance of VM live mi-gration: i) improve the algorithm for live migration used by the hypervisors, e.g., by compressing the memory pages during migration, and ii) improve the performance of the migration on the network level [30]. Although research focusing on the former solution abound [15,17,37,51,52,55,58], the impact of DC network in conducting a seamless migration is less investigated. Support-ing seamless live VM migration poses several important networkSupport-ing challenges which are discussed in the following sections.
3.1.1 Retain Network Connectivity
The ability to maintain ongoing connections is a prerequisite for seamless VM migration. This goal is easy to achieve when the VM migrates between two physical servers inside a DC where an overlay technology covers the whole network. However, different network settings (e.g., different IP address space) in remote sites make it difficult to seamlessly transfer active network con-nections. In this regard, various solutions have been proposed to help a mi-grating VM to maintain its ongoing connections. Mobile IP-based solution-s [28, 42] have been proposolution-sed in [26, 48] to addresolution-ssolution-s thisolution-s problem. Bradford et al. [15] used a combination of dynamic DNS and IP tunneling to maintain the network state during long-haul live migration. Alternative solutions such as
10 Introductory Summary
legacy L2VPN technologies, overlay networks, and SDN-based methods have been deployed in other studies [14,24,36,43,55] to address the same problem. However, there are problems with the applicability of the proposed so-lutions. For instance, mobile IP-based solutions cause triangular routing, re-quire the VM or its corresponding hypervisor to have a modified protocol stack, or need all involved networks to support a specific protocol. Extending an overlay network from one DC to another DC is neither scalable nor effi-cient as it extends the broadcast domain. Moreover, the DC administrators may need to deploy different overlay technologies in their remote sites. Lega-cy L2VPN solutions are limited in terms of redundanLega-cy, scalability, flexibility, and forwarding policies. Finally, proposed SDN-based solutions maps the old network addresses to new ones to maintain the ongoing connections which is not a scalable solution. Furthermore, the SDN-based solutions assume that the controller has a holistic view over all DC networks. Nonetheless, due to the constraint of security policies and scalability requirements, DCs usually have their own controller.
3.1.2 Large Convergence Time
Besides the system state migration, the ingress and egress traffic of the VM must also be migrated to its new location. The total time that is required for network devices to update their routing tables according to the latest changes in the network (e.g., VM migration) is known as convergence time. In the legacy networks, the procedure of network convergence is postponed to after the state migration is finished which introduces further service interruption. The ideal solution is to conduct flow migration in parallel with the system state migration as proposed in [14, 57]. The key idea in these papers is to proactively prepare the network for VM migration using an SDN-based ar-chitecture and OpenFlow-based forwarding rule rewriting. However, neither of these solutions use resiliency mechanisms to re-route the north and south traffic of the VM. Additionally, the proposed solutions are not applicable to inter-DC VM migration as different DCs usually have independent manage-ment infrastructures while these papers consider a single controller for all networks.
3.1.3 Sub-Optimal Routing
Using a sub-optimal path for egress and ingress traffic of the migrating VM is an important problem that has to be addressed. In addition to degrading the performance of the migrated application, sub-optimal routing decreases the performance of the whole network by increasing the congestion level of the links. Although the impact of sub-optimal routing in intra-DC migration is negligible, it may have a disastrous effect on inter-DC VM migration. To the best of our knowledge, sub-optimal routing problems related to inter-DC live VM migration have not been investigated previously.
Towards Seamless Live Migration in SDN-Based Data Centers 11
3.2 EVPN Automation and Management
Service deployment is one of the main concerns of network providers. Manu-al deployment of services in the network is a labor-intensive, slow, and error-prone task. Moreover, considering the size of modern DCs, manual deploy-ment of a service is not feasible. Therefore, both industry and academia put a lot of effort in developing new protocols (e.g., SNMP [16], NETCONF, etc.) and solutions (e.g., model-driven network management) to automate or partially facilitate the deployment of services in a network. However, only a few of these efforts addressed the complexity of VPNs deployment.
Authors in [32, 50, 54] propose solutions to facilitate the deployment of L3VPN and alleviate its corresponding complexities. Regarding the L2VPN solutions, the number of studies are even less. Authors in [55] utilize a cen-tral VPN controller to establish the Virtual Private LAN Service (VPLS) [29] connections between remote DCs. Likewise, authors in [31] propose an SDN-based solution to automate VPLS tunnel establishment between authorized Provider Edges (PEs). To the best of our knowledge, a framework that auto-mates the deployment of EVPN on PE routers of a DC does not exist.
Once a service is deployed in the network, the next challenge is to flexibly manage the service. The management complexity of a network technology may hamper the efficiency of provisioning that technology. This fact is par-ticularly true for MPLS-based VPN solutions as a high number of protocols are involved which make the management procedure of VPNs cumbersome. VPNService [8] is among few efforts that is developed to facilitate the man-agement of VPN services in a network. The VPNService module interacts with the OpenStack as well as other modules inside the ODL controller and improves the management of L3VPNs that are deployed in the network. To the best of our knowledge, a similar framework for EVPN does not exist.
3.3 EVPN Policy
Policies specify conditions and actions that are applied to a system in order to achieve a specific system operation goal. For instance, the network provider may desire to prioritize a specific traffic (e.g., inter-DC migration traffic) over the others during a pre-defined time interval. Ideally, network providers should be able to define the corresponding policy through high-level programming abstractions, without having to deal with the implementation complexities. Although various intent-based solutions are developed to realize this require-ment, they are limited to a number of predefined policies [49,56]. Moreover, the network actions taken by these policies are static [21, 33, 34]. Besides, routing policies for DC interconnection solutions, such as EVPN, are not addressed in any of the existing frameworks.
3.4 EVPN Scalability
Managing broadcast traffic is a key requirement for L2 technologies. Broad-cast traffic not only consumes excessive resources but also introduces security
12 Introductory Summary
vulnerabilities. For DC interconnect technologies such as EVPN, managing broadcast traffic is even more imperative as it can severely degrade the perfor-mance of inter-DC migration. However, management of broadcast traffic in EVPN technology confronts new challenges. One of the advantages of EVPN over preceding L2VPN technologies is that it provides an All-Active (A-A) mode of operation by which the traffic can truly be multi-homed. Although the A-A mode of operation is a very beneficial feature, it may introduce severe scalability problems if broadcast traffic is imported multiple times into the DC and through different paths. The router that is responsible to handle broadcast traffic is known as Designated Forwarder (DF). The default DF election algo-rithm defined by the EVPN standard [20] is called “service-carving” which is a distributed algorithm that each PE runs independently. However, service-carving encounters a number of fundamental problems such as inconsistent output, undesirable DF swap, and fairness problems. Although a number of solutions [25,39,45,47] are proposed to address the aforementioned problems, they all fail to fully address the problems.
4 Research Questions
This thesis addresses the following research questions:
RQ1: How can SDN improve live VM migration in DC networks?
To improve the live migration in DC networks, most of the research focused on enhancing migration schemes. However, live VM migration confronts a number of networking challenges that can severely degrade the performance of migration. Among all networking challenges of VM migration, two chal-lenges are considered in this thesis: i) slow network convergence, and ii) sub-optimal routing problem. We aimed at identifying, using, and extending the SDN capabilities to mitigate these problems (Paper I and Paper V).
RQ2: How can SDN automate the deployment and improve the management of DC interconnections?
To conduct inter-DC live migration, remote sites ought to be interconnect-ed. Typically, interconnecting remote sites (e.g., geo-dispersed DCs) is a very time-consuming and error-prone task as it requires significant efforts for con-figuration. The next challenge is to efficiently manage these interconnections. The ability of the SDN controller in centrally managing the network devices through various protocols motivated us to investigate an SDN-based solution to automate the deployment and improve the management of EVPN-based interconnections (Paper II).
RQ3: How can SDN improve the performance and scalability of DC intercon-nections?
Towards Seamless Live Migration in SDN-Based Data Centers 13
consequently enhance the total performance of migration. The performance of DC interconnect solutions can be improved by applying routing policies in the DC network. However, DC providers are usually unable to apply poli-cies as it requires them to have a comprehensive knowledge of network pro-tocols. On the other hand, inefficient management of broadcast traffic is a common problem for most of the network technologies which introduces se-rious scalability problems. The efficient management of broadcast traffic is of a paramount importance in EVPN technology as it provides the A-A mode of operation which leads to the broadcast traffic imported into the network through different paths. The comprehensive network view of the SDN con-troller and its close integration with DC management systems motivated us to develop an SDN-based framework that facilitates the deployment of policies (Paper III) and improves the scalability of DC interconnections built around EVPN (Paper IV).
5 Contributions
The main objective of this thesis is to improve the intra- and inter-DC live VM migrations using SDN. Additionally, this thesis proposes and develops SDN-based solutions to automate the deployment of EVPN connections among DCs, improve the management, enhance the performance, and increase the scalability of EVPN-based interconnections. These general contributions are reflected in various partial contributions made in Papers I-V, as follows: 1. An SDN-based framework for live VM migration.
Paper I and Paper V address research question RQ1. The proposed SDN-based solutions in these papers accelerate the network convergence and resolve sub-optimal routing problems for both intra- and inter-DC VM migration. In Paper I, we propose to split the intra-DC live VM migration procedure into two parts: i) a temporary local repair, and ii) a path re-optimization. In the first phase, the SDN controller proactively installs backup paths for all the ongoing connections of the migrating VM towards the new location using OpenFlow-based resiliency mechanisms. Once the VM is resumed at the new location, the SDN controller enters the second phase and removes sub-optimal routing by re-optimizing the paths.
In the inter-DC VM migration (Paper V), the controller serializes a partic-ular EVPN message, the MAC Advertisement message with MAC Mobility extended community, when the VM enters the stop-and-copy phase. This message contains information about the migration which the controller prop-agates to its peers in the control plane through MP-BGP protocol. Upon re-ceiving the message, remote DCs update their corresponding forwarding rules. This procedure reduces the migration downtime as it conducts network con-vergence in parallel with state migration. Furthermore, by deploying the gate-way functionality in the controller the inter-subnet sub-optimal routing prob-lem is addressed. Results show that the SDN-based solutions can significantly improve the performance of migration in comparison to legacy methods.
14 Introductory Summary
2. An SDN-based framework to automate the deployment and improve the man-agement of EVPN.
In Paper II, multiple modules are designed, extended, and implemented in the ODL controller to develop an SDN-based framework which automates the deployment and improves the management of EVPN-based interconnec-tions. The implemented modules use model-driven network management to automate the deployment of EVPN instances on PE routers of the DC. Addi-tionally, the controller integration with OpenStack and its capability in under-standing EVPN related messages help the controller to collect a comprehen-sive information about EVPN instances deployed in the DC network. There-fore, the DC administrator can retrieve information about EVPN instances through northbound APIs and manage the instances through high-level com-mands without being involved in the complexity of the underlying network. It is worth mentioning that the developed framework also mitigates existing problems such as ARP flooding and silent host problem3within the DC
net-works. This framework contributes to address the research question RQ2 and provides the baseline needed to obtain other objectives in Paper III, Paper IV, and Paper V.
3. An SDN-based framework to improve the performance and scalability of EVPN. Improving the quality of EVPN-based interconnections (RQ3) is the main motivation for Paper III and Paper IV. Paper III presents a policy-based framework to flexibly manage and deploy routing policies for EVPN-based interconnections. The main motivation of this paper is to help DC providers to deploy various policies, e.g., traffic engineering policies, for EVPN-based interconnections without being involved in network complexities. Paper IV proposes an SDN-based solution to improve the management of broadcast traffic in EVPN-based interconnections. The proposed solution in this paper mitigates the problem of standard methods in dealing with multi-destination traffic and further increases the scalability of EVPN technology.
6 Research Methodology
The research work described in this thesis follows the traditional scientific ap-proach in experimental computer science [19] which includes an iterative cy-cle of literature review, problem formulation, hypothesis building or descrip-tion, verificadescrip-tion, and analysis. Typically, hypothesis verification methods in-clude analytical model, simulation, real-world measurements, and emulation. All of these methods have their respective strengths and weaknesses.
An analytical method, mathematically models a system under investiga-tion, and can provide a quick insight into the overall behavior of the system. However, the analytics results can be less accurate in comparison with other
Towards Seamless Live Migration in SDN-Based Data Centers 15
methods. Computer simulations, on the other hand, involve more detailed features of the underlying system but it is often based on many assumptions and artificial modeling in order to reach a certain realistic degree. As a result, if the utilized model ignores a critical behavior of the system, and improperly handles initial conditions it may lead to incorrect conclusions. Furthermore, a considerable effort is needed to write and debug a reasonably sized simula-tion program. Although difficult to design and expensive to deploy, real-world measurement represents the lowest level of abstraction compared to analyti-cal and simulation methods. Finally, we have emulation, which is a hybrid approach between the simulation and experimentation. In emulation, some components of the experimental setup are abstracted and some components run within a real environment.
In this thesis, both emulation and real-world measurements are used to val-idate hypotheses. Although we are aware that emulation-based measurements do not allow to understand all the implications of a real-life situation such as the variable VM downtime, the emulation-based measurements are more pre-vailed. The main reason is that conducting a real VM migration, for instance, long-haul migration, requires significant infrastructure such as a distributed OpenStack environment, EVPN-VXLAN capable routers, etc., to which we do not have access.
An emulation approach is utilized to address the RQ1 and RQ3 using the well-known CORE [2] and Mininet [4] network emulators. These emulators provide us with an opportunity to evaluate the efficiency of our proposed solutions in network topologies similar to real DC networks. The experiment results are presented in Paper I, Paper III, Paper IV, and Paper V.
To tackle RQ2, we developed several modules inside the ODL controller. The performance of the implemented modules are evaluated within the fol-lowing ways: 1) a black-box test, and 2) two white-box tests. The black-box test and the first white-box test are real-world measurements. The second white-box test, on the other hand, is an example of an emulation-based mea-surement. While the black-box test evaluates the performance of the whole controller, white-box tests evaluate the performance of specific modules inside the controller.
To conduct the black-box and the first white-box tests, the Bagpipe soft-ware router [1] is extended to generate EVPN workloads in three modes in-cluding: burst, one-by-one, and single workload. For the black-box test, the performance of the whole controller is evaluated when the Bagpipe router operates in one-by-one and burst modes. In the first white-box test, the per-formance of the modules we added to the controller is evaluated by measuring the processing time of EVPN messages when the Bagpipe router operates in the one-by-one mode.
In the second white-box test, the time consumed by each module inside the controller to initialize and deploy EVPN instances in the DC network is evaluated while the controller interacts with an EVPN-enabled Nokia router imported into the GNS3 [3] emulator. To conduct the same evaluation with the real-world measurement we need a router which can support a wide range
16 Introductory Summary
of protocols similar to real DC edge routers. In particular, we need an EVPN-enabled router supporting NETCONF protocols and YANG data model lan-guage to which we do not have access. The performance results are presented in Paper II.
7 Summary of Appended Papers
Paper I – Minimizing Live VM Migration Downtime Using OpenFlow based Resiliency Mechanisms
Besides the transmission of VM, its corresponding traffic has also to be re-sumed to finish the migration procedure. The time required to update the network connections further increases the service downtime. Hence, from the networking point of view, it is very important to restore connectivity as fast as possible to provide a resilient and seamless live VM migration. In this paper, we proposed several novel schemes based on SDN that allow a fast restoration of network connectivity for a VM migration within a DC. The proposed solutions include OpenFlow resiliency method, packet bicasting, and stateful forwarding. Unlike legacy networks that defer the network con-vergence to after the VM is up-and-run at the new location, in our proposed method the controller proactively exploits one of the aforementioned schemes once it is informed about the VM migration. An evaluation using SDN ex-tensions of the network emulator CORE shows that our proposals effectively reduce the downtime leading to a more seamless live VM migration.
Paper II – Automating Ethernet VPN Deployment in SDN-based Data Centers
By increasing the size of networks and emerging multi-vender environments such as DCs, it is getting difficult for infrastructure providers to deploy a service in their networks. Despite the efforts made to offer a faster config-uration of network devices, network providers are not able to deal with on-demand services. The introduction of a new customer or service requires a set of configuration procedures which involves administrators to go through a time-consuming and error-prone configuration process. The next challenge is to effectively manage the services that are deployed in the network. To manage a service in a network, the administrators ought to have extensive knowledge about network status and protocol specifics. However, the feasi-bility of this approach is questionable due to the size of modern DCs and the wide range of services and protocols deployed in DCs. This paper proposes an SDN-based framework that automates the EVPN deployment and improves its management inside DCs using OpenStack and OpenDaylight. First, the OpenDayligh controller is extended with several modules to receive high-level commands from the OpenStack and deploy EVPN instances on DC routers using YANG data model language and NETCONF protocol. Second, the close integration of the OpenDaylight controller with public cloud platforms
Towards Seamless Live Migration in SDN-Based Data Centers 17
such as OpenStack helps the controller to have a comprehensive information about the underlying network. On the other hand, the controller knows how to communicate with network devices. As a result, the SDN controller can flexibly manage various services such as EVPNs from a centralized point. The scalability analysis shows the feasibility of our proposed solution.
Paper III – Policy-based Routing and Load Balancing for EVPN-based Data Center Interconnections
Policy-based management attempts to simplify the management of DC and helps DC providers to meet the service level agreements negotiated with each end-user. However, applying policies within a DC is complex, prone to mis-configuration, and requires the administrator to have a comprehensive insight into the network status and protocol specifics. This paper presents an SDN-based framework for policy-driven DC interconnections that are built around EVPN. The framework is designed to translate routing and other traffic engi-neering policies, which are defined for EVPN instances, into an appropriate low-level network actions to meet the policy goals. The proposed framework avoids the need to hard-code the controller behavior and allows to modify the routing, multi-homing, and load balancing strategies within and across DCs. To illustrate the benefits of the presented approach, we have implemented five simple traffic engineering strategies and evaluated them in emulated intra-DC and inter-DC networks. Our evaluation results show how different traffic engineering policies lead to a different performance in terms of throughput, latency, and flow completion time.
Paper IV – SDN Enhanced Ethernet VPN for Data Center Interconnect One of the major advantages of EVPN over legacy layer 2 VPN solutions is providing an All-Active mode of operation so that the traffic can truly be multi-homed on PE routers. However, when the Customer Edge (CE) router is multi-homed to one or more PE routers, it is necessary that only one of the PE routers should forward broadcast, unknown unicast, and multicast traffic into the DC. Importing multi-destination packets through multiple routers is destructive and leads to scalability problems such as undesirable flooding, in data and control plane. This problem ought to be addressed as it may severely degrade the performance of inter-DC migration. The PE router that assumes the primary role for forwarding BUM traffic to the CE device is called the designated forwarder. The designated forwarder election algorithm defined by the EVPN standard encounters a number of fundamental problems such as inconsistent output, undesirable designated forwarder swap, and fairness problems. In this paper, we introduce an SDN-based architecture for EVPN support, where the controller selects a designated forwarder in accordance to link utilization of DC. We show how the comprehensive view over the network using the SDN architecture helps to select an appropriate designated forwarder leading to lower overhead and better performance.
18 Introductory Summary
Paper V – EVPN/SDN Assisted Live VM Migration between Geo-Distributed Data Centers
In geo-distributed DCs, multiple DC sites are interconnected over the WAN, typically using MPLS networks. In contrast to intra-DC networks, where links have typically less than one millisecond latency and 40 or 100 Gbps link capacity is common, WAN connections have significantly higher latency and lower capacity. The higher latency and lower link capacity prolongs the mi-gration downtime and seriously degrades the performance of VM mimi-gration. This paper presents a novel approach for long-haul live VM migration between geo-distributed DCs that accelerates the network convergence and optimizes the post-migration traffic routing. First, the controller reduces the network convergence time by pre-advertising the migration when the VM enters the stop-and-copy phase. To do so, the controller serializes and propagates an ap-propriate EVPN message in the control plane using MP-BGP protocol. Upon receiving this message, the peers of the controller in remote DCs start updat-ing the flow tables inside their domain. As a result, the network starts con-vergence while the state migration is in progress. Second, the SDN controller resolves the sub-optimal routing problem that arises as a result of migration implementing a distributed anycast gateway. By performing experiments in emulated scenarios, we find that our approach significantly reduces the down-time compared to alternative schemes, particularly when the latency between DCs is higher. Furthermore, addressing the sub-optimal routing problem re-markably increases the performance of migrating VM.
8 Conclusions and Future Work
Live VM migration is a promising solution for data center administrators to achieve a wide range of objectives, from load balancing to disaster evacuation. Although many solutions have been proposed to improve the VM migration schemes, the networking aspects of live VM migration are mainly overlooked. The work presented in this thesis investigates the networking challenges of VM migration, in particular, slow network convergence and sub-optimal rout-ing problem, and proposes SDN-based solutions to improve the intra- and inter-DC migration procedure.
To conduct inter-DC migration remote sites ought to be interconnected. The EVPN is the interconnection technology that is used in this thesis due to its outstanding features. Automating the deployment, improving the manage-ment, enhancing the performance, and increasing the scalability of EVPN-based interconnections are other objectives that are investigated in this the-sis. We developed several modules inside the ODL controller to automate the EVPN interconnection deployment on DC edge routers using Yang data mod-el language and NETCONG protocol. Further, we extended the controller to improve the management and enhance the quality of EVPN-based intercon-nections. Table 1 summarizes the research objectives of this thesis and how these objectives are evaluated.
Towards Seamless Live Migration in SDN-Based Data Centers 19
To have a seamless migration addressing the system and networking chal-lenges of migration is crucial. Nonetheless, scheduling the migration is of the same importance. Migration of a VM regardless of its system and the under-lying network states may lead to severe service interruptions. This problem is exacerbated when applications that are running on multiple VMs construct a service chain. In such a scenario, migration of the VM can significantly de-crease the performance of the whole chain. In a broader picture, any change in the current state of the network ought to be carefully arranged otherwise, it may lead to notable service disruptions. Principally, scheduling reconfigu-ration in a network helps administrators to achieve their goals, for instance to reach an energy-efficient state, while the negative impacts on the network and applications during the reconfiguration are minimized. Knowing the cost of migration and its impact on the network, we intend to evaluate the reconfig-uration costs of service chains and propose solutions to minimize them in the future extension of this work.
20 Introductory Summary
Table 1: Summary of research questions, objectives, and methods.
RQ 1 RQ 2 RQ 3 R esear ch Objectiv e
Reduce migration down-time. [Paper I, Paper V] Resolve sub-optimal rout-ing problem. [Paper V]
Automate the deployment of EVPN. [Paper II] Improve the management of EVPN. [Paper II]
Improve the performance of EVPN-based intercon-nections. [Paper III] Enhance the scalability of EVPN-based interconnec-tions. [Paper IV]
Emulation
DC topology and back-ground traffic in CORE network emulator. [Paper I]
DC topology in Mininet network emulator. [Paper V]
Black-box test: Interaction of ODL controller with EVPN-enabled router im-ported into the GNS3 em-ulator. [Paper II]
DC topology and background traffic in CORE network emulator. [Paper III]
Emulation of DF swap functionality. [Paper IV] DC topology in Mininet network emulator. [Paper IV] R eal-W or ld Measur ement —
White-box and Black-box test: Add EVPN capabil-ity to Bagpipe router to generate artificial EVPN workloads. [Paper II]
— Im plementation & Measur ement Implementation of SDN-based resiliency mecha-nisms. [Paper I] Extension of ODL mod-ules which are developed in Paper II. [Paper V] Measurement of down-time and throughput. [ Paper I, Paper V] Measurement of FCT. [Paper V]
Implementation of several new modules inside ODL controller. [Paper II] Measurement of con-troller performance in deploying EVPNs and processing EVPN related messages. [Paper II]
Implementation of traffic engineering routing poli-cies. [Paper III]
Measurements of port utilization of DC edge switches, FCT, and RTT. [Paper III]
Measurements of packet loss percentage and num-ber of received broadcast traffic. [Paper IV]
FCT = Flow Completion Time. RTT = Round Trip Time. DF = Designated Forwarder.
Towards Seamless Live Migration in SDN-Based Data Centers 21
References
[1] BaGPipe: A lightweight implementation of BGP VPNs. https:// github.com/Orange-OpenSource/bagpipe-bgp/.
[2] Core: Common open research emulator. https://www.nrl.navy. mil/itd/ncs/products/core.
[3] Gns3: Graphical network simulator. https://www.gns3.com/. [4] Mininet. http://mininet.org/.
[5] Open vSwitch: Multilayer Open Virtual Switch. https:// openvswitch.org/.
[6] OpenDaylight: Open Source SDN Platform. https://www. opendaylight.org/.
[7] OpenStack. https://www.openstack.org//.
[8] VPNService. https://github.com/opendaylight/netvirt/tree/ master/vpnmanager.
[9] S. Akoush, R. Sohan, A. Rice, A. W. Moore, and A. Hopper. Predict-ing the performance of virtual machine migration. In IEEE Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), Miami, Fl, USA, Aug. 2010.
[10] Y. Ben-Itzhak, K. Barabash, R. Cohen, A. Levin, and E. Raichstein. En-forSDN: Network policies enforcement with SDN. In IFIP/IEEE Sym-posium on Integrated Network Management (IM), Ottawa, ON, Canada, May 2015.
[11] G. Bianchi, M. Bonola, A. Capone, and C. Cascone. OpenState: Programming platform-independent stateful OpenFlow applications in-side the switch. ACM SIGCOMM Computer Communication Review, 44(2):44–51, Aug. 2014.
[12] M. Bjorklund. YANG - a data modeling language for the Network Con-figuration Protocol (NETCONF). RFC 6020, Oct. 2010.
[13] D. L. Black, Z. Wang, M. A. Carlson, W. Weiss, E. B. Davies, and S. L. Blake. An architecture for differentiated services. RFC 2475, Dec. 1998. [14] B. Boughzala, R. B. Ali, M. Lemay, Y. Lemieux, and O. Cherkaoui. OpenFlow supporting inter-domain virtual machine migration. In IEEE Conference on Wireless and Optical Communications Networks (WOCN), Paris, France, May 2011.
22 Introductory Summary
[15] R. Bradford, E. Kotsovinos, A. Feldmann, and H. Schiöberg. Live wide-area migration of virtual machines including local persistent state. In ACM Conference on Virtual Execution Environments (VEE), San Diego, CA, USA, June 2007.
[16] J. Case, M. Fedor, M. Schoffstall, and J. Davin. Simple Network Man-agement Protocol (SNMP). RFC 1157, May 1990.
[17] K. Chanchio and P. Thaenkaew. Time-bound, thread-based live migra-tion of virtual machines. In IEEE/ACM Symposium on Cluster, Cloud and Grid Computing (CCGrid), Chicago, IL, USA, May 2014.
[18] C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live migration of virtual machines. In USENIX Sym-posium on Networked Systems Design & Implementation (NSDI), Berkeley, CA, USA, May 2005.
[19] P. J. Denning. What is experimental computer science? Communications of the ACM, 23(10):543–544, Oct. 1980.
[20] J. Drake, W. Henderickx, A. Sajassi, R. Aggarwal, D. N. N. Bitar, A. Isaac, and J. Uttaro. BGP MPLS-Based Ethernet VPN. RFC 7432, Feb. 2015.
[21] X. Du, Z. Lv, J. Wu, C. Wu, and S. Chen. PDSDN: A policy-driven SDN controller improving scheme for multi-tenant cloud datacenter environ-ments. In IEEE Conference on Services Computing (SCC), San Francisco, CA, USA, June 2016.
[22] R. Enns, M. Bjorklund, J. Schoenwaelder, and A. Bierman. Network Configuration Protocol (NETCONF). RFC 6241, June 2011.
[23] A. Ford, C. Raiciu, M. J. Handley, and O. Bonaventure. TCP extensions for multipath operation with multiple addresses. RFC 6824, Jan. 2013. [24] F. Hao, T. V. Lakshman, S. Mukherjee, and H. Song. Enhancing
dynam-ic cloud-based servdynam-ices using network virtualization. In ACM Workshop on Virtualized Infrastructure Systems and Architectures (VISA), Barcelona, Spain, Aug. 2009.
[25] W. Hao, L. Yong, and Q. Liang. Handshaking mechanism for DF elec-tion, May.
[26] E. Harney, S. Goasguen, J. Martin, M. Murphy, and M. Westall. The efficacy of live virtual machine migrations over the internet. In ACM Workshop on Virtualization Technology in Distributed Computing (VTD-C), Reno, NV, USA, 2007.
[27] M. R. Hines, U. Deshpande, and K. Gopalan. Post-copy live migration of virtual machines. ACM SIGOPS operating systems review, 43(3):14–26, July 2009.
Towards Seamless Live Migration in SDN-Based Data Centers 23
[28] D. B. Johnson, J. Arkko, and C. E. Perkins. Mobility support in IPv6. RFC 6275, Jul. 2011.
[29] K. Kompella and Y. Rekhter. Virtual Private Lan Service (VPLS) using BGP for auto-discovery and signaling. RFC 4761, Jan. 2007.
[30] J. Liu, Y. Li, and D. Jin. SDN-based live VM migration across datacen-ters. In ACM SIGCOMM, Chicago, IL, USA, Aug. 2014.
[31] M. Liyanage, M. Ylianttila, and A. Gurtov. Improving the tunnel man-agement performance of secure VPLS architectures with SDN. In IEEE Annual Consumer Communications Networking Conference (CCNC), Las Vegas, NV, USA, Jan. 2016.
[32] G. Lospoto, M. Rimondini, B. G. Vignoli, and G. D. Battista. Rethink-ing virtual private networks in the software-defined era. In IFIP/IEEE Symposium on Integrated Network Management (IM), Ottawa, ON, Cana-da, May 2015.
[33] C. C. Machado, L. Z. Granville, A. Schaeffer-Filho, and J. A. Wick-boldt. Towards SLA policy refinement for QoS management in software-defined networking. In IEEE Conference on Advanced Information Net-working and Applications (AINA), Victoria, BC, Canada, May 2014. [34] C. C. Machado, J. A. Wickboldt, L. Z. Granville, and A. Schaeffer-Filho.
Arkham: an advanced refinement toolkit for handling service level agree-ments in software-defined networking. Journal of Network and Computer Applications, 90:1 – 16, 2017.
[35] M. Mahalingam, D. Dutt, K. Duda, P. Agarwal, L. Kreeger, T. Sridhar, M. Bursell, and C. Wright. Virtual eXtensible Local Area Network (VXLAN): A framework for overlaying virtualized layer 2 networks over layer 3 networks. RFC 7348, Aug. 2014.
[36] V. Mann, A. Vishnoi, K. Kannan, and S. Kalyanaraman. Crossroad-s: Seamless VM mobility across data centers through software defined networking. In IEEE Network Operations and Management Symposium (NOMS), Maui, HI, USA, Apr. 2012.
[37] A. J. Mashtizadeh, M. Cai, G. Tarasuk-Levin, R. Koller, T. Garfinkel, and S. Setty. XvMotion: Unified virtual machine migration over long distance. In USENIX Conference on Annual Technical Conference (USENIX ATC), Philadelphia, PA, USA, June 2014.
[38] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S. Shenker, and J. Turner. OpenFlow: Enabling innovation in campus networks. ACM SIGCOMM Computer Communication Review, 38(2):69–74, Mar. 2008.
[39] S. Mohanty, K. Patel, A. Sajassi, J. Drake, and T. Przygienda. A new designated forwarder election for the EVPN, Apr. 2017.
24 Introductory Summary
[40] R. Nasim and A. J. Kassler. Network-centric performance improve-ment for live VM migration. In IEEE Conference on Cloud Computing (CLOUD), New York, NY, USA, June-July 2015.
[41] ONF. Software-Defined Networking: The new norm for networks. ON-F White Paper, Apr. 2012.
[42] C. E. Perkins. IP mobility support for IPv4. RFC 3344, Aug. 2002. [43] P. Raad, S. Secci, D. C. Phung, A. Cianfrani, P. Gallard, and G. Pujolle.
Achieving sub-second downtimes in large-scale virtual machine migra-tions with LISP. IEEE Transacmigra-tions on Network and Service Management, 11(2):133–143, June 2014.
[44] J. Rabadan, S. Sathappan, W. Henderickx, A. Sajassi, and J. Drake. Inter-connect solution for EVPN overlay networks. Internet-draft, Internet Engineering Task Force, Mar. 2018.
[45] J. Rabadan, S. Sathappan, T. Przygienda, W. Lin, J. Drake, A. Sajassi, and S. Mohanty. Preference-based EVPN DF election, Dec. 2016. [46] S. Sahni and V. Varma. A hybrid approach to live migration of virtual
machines. In IEEE Conference on Cloud Computing in Emerging Markets (CCEM), Bangalore, India, Oct. 2012.
[47] A. Sajassi, G. Badoni, D. Rao, P. Brissette, and J. Drake. Fast recovery for EVPN DF election, Mar. 2017.
[48] E. Silvera, G. Sharaby, D. Lorenz, and I. Shapira. IP mobility to support live migration of virtual machines across subnets. In ACM Conference on The Israeli Experimental Systems, Haifa, Israel, May 2009.
[49] M. Stein, M. Scharf, and V. Hilt. SDN policy-driven service chain place-ment in OpenStack. In IFIP/IEEE Symposium on Integrated Network and Service Management (IM), Lisbon, Portugal, May 2017.
[50] K. Suzuki and H. Kaneko. An openflow controller for reducing opera-tional cost of IP-VPNs. Technical report, Apr. 2014.
[51] P. Svärd, B. Hudzia, S. Walsh, J. Tordsson, and E. Elmroth. The noble art of live vm migration-principles and performance of pre copy, post copy and hybrid migration of demanding workloads. Technical report, 2014.
[52] P. Svärd, J. Tordsson, B. Hudzia, and E. Elmroth. High performance live migration through dynamic page transfer reordering and compres-sion. In IEEE Conference on Cloud Computing Technology and Science (CLOUDCOM), Athens, Greece, Nov. 2011.
[53] B. J. Van Asten, N. L. Van Adrichem, and F. A. Kuipers. Scalability and resilience of software-defined networking: An overview. arXiv preprint arXiv:1408.6760, 2014.
