Location-based Search
Service for a P2P OpenStack
System
TONY THOMAS
KTH ROYAL INSTITUTE OF TECHNOLOGY
Location-based Search
Service for a P2P OpenStack
System
TONY THOMAS
Master in ICT Innovation Date: October 24, 2018
Industry Supervisor (s): João Monteiro Soares, Fetahi Wuhib Academic Supervisor: Prof. Rolf Stadler
Examiner: Prof. Viktoria Fodor
Swedish title: Lägesbaserad Söktjänst för ett Peer-to-Peer OpenStack System
iii
Abstract
Sammanfattning
I det här examensarbetet designas och utvecklas en mekanism som möjligör sökning efter Peer-to-peer noder baserat på deras geografiska position. I en peer-to-peer node skapar och underhåller mekanismen ett distribuerat sökindex. I arbetet introduceras ett bootstrapping (eller uppstarts-) protokoll, som vi kallar NS-Bootstrap, som är ansvarigt för att bygga sökindexet för en peer-to-peer-nod. Vi inför också ett sökpro-tokoll som använder detta index, NSSearch. Mekanismen har testats i en emulerad peer-to-peer-miljö med upp till 3000 noder. Den utvär-deringen tyder på att tillståndsdata som upprätthålls i en nod växer logaritmiskt med N, där N är det totala antalet peer-to-peer noder i systemet.
v
Acknowledgements
1 Introduction 1 1.1 Problem presentation . . . 2 1.2 Approach . . . 2 1.3 Contribution . . . 3 1.4 Outline . . . 3 2 Related Research 4 2.1 Flooding-based approaches . . . 4 2.1.1 Iterative deepening . . . 4 2.1.2 Echo protocols . . . 5 2.2 Tree-based approaches . . . 6 2.2.1 GAP protocol . . . 6 2.3 Index-based approaches . . . 7
2.3.1 Local index based search . . . 7
2.3.2 Routing index based search . . . 8
2.4 Summary . . . 8
3 Background 9 3.1 Peer to Peer (P2P) systems . . . 9
3.2 Membership management in P2P systems . . . 10
3.2.1 CYLCON protocol . . . 11
3.3 P2P architecture for OpenStack . . . 12
3.3.1 OpenStack . . . 12
3.3.2 P2P OpenStack architecture . . . 13
3.3.3 P2P OpenStack system: Scheduling a workload . 14 3.4 Summary . . . 14
4 Location-based P2P lookup 15 4.1 Search architecture . . . 15
4.2 Abstract Namespace Tree (ANT) . . . 17
CONTENTS vii
4.2.1 Tracking the state of an ANT . . . 19
4.2.2 Use case: Geographical locations . . . 21
4.3 The query language . . . 22
4.4 NSBootstrap: Node bootstrap protocol . . . 22
4.4.1 Using CYCLON for finger table maintenance . . . 27
4.4.2 Stable state for NSBootstrap . . . 30
4.4.3 Optimize NSBootstrap resource utilization . . . . 30
4.4.4 Factors affecting NSBootstrap convergence time . 31 4.5 NSSearch: The lookup protocol . . . 32
4.5.1 Iterative traversal of an ANT by NSSearch . . . . 34
4.6 Summary . . . 36
5 Evaluations and discussions 37 5.1 Test Setup . . . 37
5.2 NSBootstrap . . . 40
5.2.1 Metrics measured . . . 40
5.2.2 Validation . . . 40
5.2.3 Evaluation . . . 40
5.2.4 Results and discussion . . . 41
5.2.5 Conclusions . . . 51
5.3 NSSearch . . . 52
5.3.1 Metrics measured . . . 53
5.3.2 Validation . . . 53
5.3.3 Evaluation . . . 53
5.3.4 Results and discussion . . . 54
5.4 Summary . . . 56 6 Conclusions 58 6.1 Future Work . . . 58 6.2 Experiences . . . 59 Bibliography 61 A Sample Code 66 A.1 Sample NSBootstrap operation . . . 66
A.2 Sample NSBootstrap implementation . . . 67
A.3 Sample CYCLON implementation . . . 68
A.4 Finding next prefix of a label . . . 70
1.1 The problem addressed by this thesis work. A workload scheduling problem is translated to a resource lookup problem. Users can later set up their workload on the k nodes returned. . . 2
2.1 Tree-based approaches for resource lookup build an in-dex for a node with pointers to its parent, peers, and children. . . 6
3.1 P2P Architecture with OpenStack instances sharing an identity service. Image is courtesy of Xin et al.[2]. . . 13
4.1 Search architecture, figure inspired from [21] . . . 16
4.2 The lifetime of a node executing our solution. CYCLON protocol is an integral part of our solution. . . 16
4.3 An ANT formed out of consistent labeling extending basePrefix root. Physical P2P nodes are at the leaf level (shaded gray). . . 18
4.4 A P2P node should keep track of at least one node in all childPrefixes of all its prefixLevels. In effect, node4 should keep track of at least one node on all branches of the ANT which includes a shaded token (along with its leaf sibling, node2). . . 20
4.5 ANT formed based on geographical location. This is proposed as a solution for our P2P OpenStack location-based search problem. . . 21
5.1 Sample ANT state for various configurations of D, K and L . . . 39
LIST OF FIGURES ix
5.2 Evaluation results for a system with fixed K and L val-ues with Depth D. . . 42
5.3 Evaluation results for a system with fixed D and L val-ues with K value changed for 10x runs. . . 45
5.4 Evaluation results for a system with fixed D and K val-ues with the number of Leaf P2P nodes L changed be-tween iterations. . . 48
5.5 CPU and Memory usage by a sample N + 1th node on executing NSBootstrap. The annotation points to the moment in time when the node attained a NSBootstrap stable state. . . 49
5.6 The percentage of Ntotal a node n should keep track of while executing NSBootstrap . . . 51
5.7 Number of total search iterations in the system for a searchPrefix of depth d and count c. The iterations for count=1 overlap the results for count=2. . . 54
4.1 childPrefixes for all prefixLevels of root.a.b.node4 from the sample tree in Figure4.3. The prefixLevel root.a.b. does not have a childPrefix as it does not have any non-leaf nodes on the ANT. . . 19
4.2 Description and properties of key terms used through-out this report. . . 21
4.3 Utility functions used in Algorithm1. . . 25
4.4 Finger table of node root.a.b.node4 from Figure4.3after execution of NSBootstrap lines 3-6. . . 26
4.5 Finger table of node root.a.b.node4 from Figure4.3after execution of NSBootstrap lines 7-8. . . 26
4.6 Finger table of node root.a.b.node4 from Figure4.3after several CYCLON shuffling rounds at root. prefixLevel . . 27
4.7 The search index built by NSBootstrap (a), and the ac-tual finger tables built and maintained by CYCLON (b-d) for root.a.b.node4 from Figure4.3. . . 29
4.8 Utility functions used in NSSearch iterative implemen-tation (Algorithm2). . . 35
5.1 Various configurations of D, K and L used for our ex-periments to evaluate NSBootstrap . . . 41
5.2 Sample search queries built to evaluate NSSearch by chang-ing searchPrefix and count. . . 54
5.3 NSSearch iteration steps executed by node10 to find 4 fingers in root.a.a.b.b.a.b. searchPrefix. These 7 steps are responsible for the maximum value for d=6,c=4 in Figure5.7. . . 56
Chapter 1
Introduction
OpenStack [1] is a set of software tools for cloud computing, popular in both industry and academia. It is mostly deployed as an Infrastructure as a Service (IaaS) platform. When deployed as an IaaS, OpenStack acts as a cloud Operating System managing large pools of computing, storage and networking resources. One of the primary functionali-ties of OpenStack is to schedule Virtual Machines (VMs), among other workloads. The academia has been investigating architectures to scale up OpenStack both (1) vertically, by supplying more compute, storage and networking resources to a single OpenStack instance and (2) hori-zontally, by stacking multiple OpenStack instances sharing workloads. The P2P OpenStack architecture by Xin et al. [2], is a step in the latter direction.
In general, P2P architectures are attractive for scaling up large sys-tems. Namely, they have a low set-up barrier and offer a possibility to aggregate resources. Moreover, their distributed nature provides resilience against faults and brute force attacks.
The proposed P2P OpenStack system by Xin et al. involves a mes-sage relaying service called an agent residing on top of an OpenStack instance. An agent along with its corresponding OpenStack instance forms an OpenStack cloudlet. A user interacts with an agent in the sys-tem and attains a single point of view of the whole infrastructure. Xin et al. designed and developed a working implementation of the archi-tecture and found improvements in resource utilization and service time for scheduling workloads [2].
Scheduling a workload in the proposed P2P OpenStack system ar-chitecture involves selecting multiple OpenStack cloudlet(s), followed
by a filtering based on their physical properties. Modern technologies like 5G, Edge and Fog computing require advanced selection and fil-tering of these cloudlets, say, based on their geographic location. This use case is addressed by this thesis work.
1.1
Problem presentation
Modern technologies like 5G, Edge and Fog computing require work-loads to be deployed at specific geographical locations [3, 4]. Hybrid clouds [5] also require deploying a workload at a particular location. For a cloud service customer, scheduling a workload at a specific lo-cation can be beneficial as it: (1) allows to set up services according to the data privacy laws of a region (2) helps in satisfying Quality of Service requirements. The P2P OpenStack architecture was designed to satisfy these requirements.
This thesis addresses the requirement of a P2P OpenStack system to schedule a workload at a given location. A rough overview of the addressed problem, and how scheduling a workload translates to a search problem is depicted in Figure1.1below.
Figure 1.1: The problem addressed by this thesis work. A workload scheduling problem is translated to a resource lookup problem. Users can later set up their workload on the k nodes returned.
1.2
Approach
CHAPTER 1. INTRODUCTION 3
A centralized solution using a registry is a simple approach to the problem addressed by the thesis. Popular services like Domain Name System (DNS) already employ a centralized approach. We do not con-sider a centralized solution approach, since
1. Centralized solutions create bottlenecks or single point of fail-ures in the system.
2. A core driving principle behind the P2P architecture for Open-Stack is to avoid any centralized functionality [2]. Hence, adding a centralized component defeats the purpose of the architecture.
1.3
Contribution
The major contributions of this thesis work are:
1. Design and development of algorithms to solve the location-based search problem. This involves an algorithm to (1) build a search index for a participating node and (2) execute a search traversing through this index.
2. Validation of the solution and evaluation of its impact on partici-pating nodes using an emulated testbed of up-to 3000 P2P nodes.
1.4
Outline
Related Research
The location-based workload scheduling requirement of the P2P Open-Stack system translates to a P2P resource lookup problem. Resource lookup in a P2P network is a well-known engineering problem. Li et al. list multiple approaches for solving the search problem using cen-tralized, structured, unstructured and hybrid P2P networks [6]. In this section, we detail multiple of such techniques based on unstructured P2P systems.
2.1
Flooding-based approaches
A flooding-based approach involves a node broadcasting a query to all (or a subset) of its peers in a P2P network and later collecting back the results. Flooding is effective in guaranteeing a result (if it exists in the system). However, it creates a large number of messages in the system [6]. Li et al. describe multiple optimizations of flooding-based P2P search to control the degree of flooding and thereby, reducing the number of messages in the network. Multiple P2P search implemen-tations based on flooding are listed below. For these methods, it is expected that a node in a P2P system knows about at least one other of its peers (neighbors).
2.1.1
Iterative deepening
Iterative deepening involves repeated Breadth First Search (BFS) traver-sal through a P2P system while incrementing the depth of travertraver-sal after each failed try [7]. The depth of traversal is controlled by depth
CHAPTER 2. RELATED RESEARCH 5
limit parameter. A query is terminated when the search is answered, or if the depth limit is met. While the depth limit is not met, nodes re-transmit the message to other nodes in the system if the query is not answered.
Iterative deepening is a controlled flooding approach. However, it can result in multiple duplicate messages in the system. Iterative deepening is a possible solution for the use case addressed by this the-sis work. However, the uncertainties connected with (1) the number of messages in the system, and (2) the execution time, drive us to look for alternatives.
2.1.2
Echo protocols
Introduced by Segall et al., the Echo protocol introduces controlled ex-pansion and contraction of messages in P2P networks [8]. For a P2P resource lookup, Echo can be useful to flood the network in a deter-ministic fashion and get back the result at the querying node. The algorithm describes two phases for a message in a P2P system: (1) the expansion phase and (2) the contraction phase.
During the expansion phase, Echo builds a minimum spanning tree rooted at the node that received a lookup query. The root node creates and forwards an echo message to all its neighbors. When a node re-ceives an echo message, it (1) stores a pointer to the sender as its parent (2) starts local operations as instructed by the echo message and (3) forwards the echo message to all its neighbors except its parent. Later, these pointers are used to build a minimum spanning tree.
The contraction phase starts once the query reaches a leaf node in the P2P system. In other words, when a node receives an echo message and it does not have anybody else in its neighbor list to forward it to, the contraction phase starts for that echo message. The leaf node replies to its parent node with results from its local operation. This message is propagated all the way back to the root node. In general, the amount of information processed is more at nodes closer to the root node.
2.2
Tree-based approaches
A Tree-based solution for P2P search requires a participating node to maintain a nominal amount of pointers to other nodes in the system. These pointers can be to a relative parent, peer(s) or children node(s) in the network graph of the system. Even though unstructured P2P systems have a flat structure, these pointers create a tree on top of the P2P system. An example is described in Figure2.1.
Figure 2.1: Tree-based approaches for resource lookup build an index for a node with pointers to its parent, peers, and children.
Tree-based approaches are found to be more effective than flooding-based approaches, as a node (1) needs not to flood the entire network to answer a query, and (2) has to store only a finite (and constantly changing) amount of information about a set of other nodes (namely parent, peers, and children). An implementation of the tree-based ap-proach, and how it can solve the location-based lookup problem is given below.
2.2.1
GAP protocol
CHAPTER 2. RELATED RESEARCH 7
For a given system property F agreed throughout the system, a participating node k with children 1..n stores the result of Pn
i=1F i. This aggregate provides a node, an overview of the properties shared by its children on a link. This information is shared with its neighbors on the GAP tree. An example of this function F can be the current aggregate CPU load on an interface, network usage on a link, etc.
A change in the value of F at a node level is propagated to all neighbors of a node. A P2P node updates its parent node about this change. Peers of the node in the network graph will then receive this updated information. Hence, GAP is an ideal solution to keep track of (system properties of) a constantly changing P2P network.
For the location-based search problem addressed in this thesis, GAP is a promising solution. Physical resources such as processing power and memory at a link can be aggregated. However, locally caching all geographical locations covered by a link is a challenging task, es-pecially with memory and storage limitations at a node level. More-over, expressing geographical locations as a mathematical function F is equally challenging.
2.3
Index-based approaches
Index-based approaches build a lookup index per node. An example is a database which stores information about the location of user data in a P2P system. This information can be populated actively over successful communications and queries, or passively on analyzing past request-response paths. Two implementations using an index-based approach, and how they can be possible solutions for the location-based lookup problem addressed by this thesis work is presented below.
2.3.1
Local index based search
Yang et al. (2002) describe an index-based approach in which a node stores enough information about other nodes within a k-hop distance from it [7] . Nodes also agree on maximum depths (on a network-graph of the P2P system) at which a query should be processed. Nodes at depths different than these will just forward the query and not pro-cess it, termed node-skips.
location-based lookup problem addressed by this thesis work, building a local search index per node is a promising option. However, for a node, the search index will have to maintain information about the geographic locations covered by all nodes within a k-hop distance. Moreover, the node skips affect the accuracy of the solution.
2.3.2
Routing index based search
Introduced by Crespo et al. (2002), the Routing index is similar to the Local index (Section2.3.1) [11]. Routing index based search requires a node to store (1) information about its immediate 1-hop routes, and (2) hints of resources these hops can serve. This allows a node to decide correctly on request-routing while handling a search query. This results in a controlled forwarding of messages in the system.
For the location-based search problem addressed by this thesis, a routing index per node results in a controlled forwarding of messages in the system. However, geographic location has a complicated rep-resentation (address fields, state, country, continent, etc). Unless all participating nodes know how to derive routing decisions using these representations, a single hop routing information alone makes it diffi-cult to route a location-based search request.
2.4
Summary
This chapter introduced multiple research works that solve the P2P search problem. These are possible solutions to our location-based search problem. In general, we find that (1) nodes should agree on a representation of geographical locations throughout the system, and (2) participating nodes need to store an amount of state information to avoid flooding the system.
Chapter 3
Background
In this chapter, we present background information related to P2P sys-tems, their classification, and functionalities like membership man-agement. We also talk about the CYCLON protocol, which plays a fundamental part in our solution. We also provide background in-formation into OpenStack and the P2P OpenStack system architecture introduced by Xin et al. We also detail how workload-scheduling is currently handled by a P2P OpenStack system.
3.1
Peer to Peer (P2P) systems
P2P systems are widely studied and are alternatives to centralized sys-tems. It involves loosely or strictly connected nodes participating in system related tasks.
Ou et al. suggest that the origin of P2P technologies can be dated back to the 1960s [12]. Since then, P2P technologies have been adopted widely for distributed file sharing, torrents, etc. Multiple benefits of a P2P system, when compared to a traditional centralized system are (from Ou et al.):
1. P2P systems are self-organizing, tolerant to frequent node addi-tion, failure and node churn.
2. P2P systems assure load balancing and decentralization. This makes them fault tolerant and resilient to Denial of Service at-tacks.
3. Nodes can be loosely co-ordinated with little to zero administra-tive arrangements.
3.2
Membership management in P2P systems
Ideally, a flat overlay network on top of physical nodes forms the P2P network for a system. Membership management protocols are respon-sible for setting up this overlay network. Hence, they are integral to the functioning of a P2P system. The following properties of a P2P system make membership management a challenging task:1. Lack of global knowledge at node level: Gossip-based dissemi-nation protocols control flooding of messages in a system by for-warding a request to a selected subset of nodes. This subset of nodes is selected from its local index. For correct results, these protocols assume that every node in the system is included in the local index list of some other node. In other words, they as-sume that the neighbor list of each node is drawn uniformly from the system. Ganesh et al. detail why this is not possible without complicated synchronization and why the size of stored state af-fects the performance of the system [13]. Hence, the lack of global knowledge at the node level makes maintaining connectivity in a P2P system challenging.
2. Flooding is not an optimal solution: Flooding can broadcast state information throughout a P2P system. It is discouraged pri-marily due to the large number of messages it creates in the P2P system. Lv et al. detail that flooding creates an imbalance in the total load on participating nodes [14].
Membership management is handled with respect to the structure of the P2P overlay. Centralized, loosely structured and unstructured are three of these P2P overlay architectures. For these three, member-ship management is handled as follows:
CHAPTER 3. BACKGROUND 11
2. Loosely structured/decentralized P2P overlays do not have a centralized meta-data store, but information is spread through-out the P2P system in an orderly fashion so that queries are sat-isfied without flooding the network. Handling node churn and failure is a challenging task in this architecture. Hybrid overlay networks are a variation of this architecture. In hybrid overlays, a set of nodes, termed super-peers or dominating nodes are respon-sible to serve for a subset of normal nodes [6].
3. Unstructured P2P overlays are of much academic interest. They form P2P systems with nodes sharing equal roles. Nodes keep track of a set of other nodes (neighbors) through a membership management protocol. Gnutella is an example of such a protocol [15]. Information about the topology and resources held is stored arbitrarily among nodes. CYCLON is an example of an unstruc-tured P2P membership protocol that maintains a random neigh-bor list. Resource lookup gets more complex in such a setting as a node has information only about a subset of its peers.
3.2.1
CYLCON protocol
CYCLON, introduced by Voulgaris et al. is a robust, scalable, decen-tralized and inexpensive membership management protocol for an un-structured P2P system [16]. The protocol is characterized by its build-ing of network graphs with low diameter, low clusterbuild-ing and highly symmetric node degrees. Each node maintains an index table with in-formation about a number of other nodes, called a neighbor table, on which CYCLON executes its shuffling algorithm. The shuffling algo-rithm is responsible for (1) introducing new nodes to the system (2) notifying participating nodes about node churns or fail events, and is detailed later in this section.
1. Increase by one the age of all neighbors.
2. Select neighbor Q with the highest age among all neighbors, and l − 1 other random neighbors. l is a system parameter, called Shuffle Length.
3. Replace Q’s entry with an entry of age 0 and with P’s address. 4. Send the updated subset to peer Q.
5. Receive from Q a subset of no more than l of its own neighbors. 6. Discard entries pointing at P and entries already contained in P’s
cache.
7. Update P’s cache to include all remaining entries, by firstly using empty cache slots (if any), and secondly replacing entries among the ones originally sent to Q.
In essence, the algorithm makes sure that every node in the sys-tem keep a minimum number of periodically varying neighbors in its neighbor table. The age parameter also makes sure that there are no circular dependencies within the system as explained and evaluated in [16].
3.3
P2P architecture for OpenStack
In this section, we introduce OpenStack, a cloud Infrastructure as a Service (IaaS). We also detail the motivations behind, and the system design of the P2P OpenStack architecture [2,17] with its shortcomings.
3.3.1
OpenStack
OpenStack is an Open Source set of tools and services which provide a framework to deploy public and private clouds. It provides a cost-efficient and stable solution for service providers to deploy and main-tain private clouds when compared to proprietary solutions like Ama-zon AWS, Google or Microsoft cloud solutions.
CHAPTER 3. BACKGROUND 13
3.3.2
P2P OpenStack architecture
Vertical scaling up of OpenStack by adding compute nodes to a stan-dalone OpenStack instance has been found to create bottlenecks as the number of compute nodes increases. Benchmarking [18] shows how the CPU and memory usage races up as a total of 1000 compute nodes are added to a single OpenStack instance. A previous study by Cisco Inc. [19] found an increased amount of failure of requests at a config-uration of ≥ 147 compute nodes per OpenStack instance. These were the motivations behind the P2P OpenStack architecture [2].
In their architecture, Xin et al. use a message proxy service called an agent, on top of individual OpenStack instances. An OpenStack instance with its corresponding agent is termed an OpenStack cloudlet. The agent overlay forms the P2P network in their architecture.
Figure 3.1: P2P Architecture with OpenStack instances sharing an identity service. Image is courtesy of Xin et al.[2].
3.3.3
P2P OpenStack system: Scheduling a workload
Xin et al. employ a random selection algorithm to select an OpenStack instance while scheduling a workload. An agent uses a power of twochoices[20] selection to decide to which (other) OpenStack instance a request gets forwarded. The method randomly selects two OpenStack instances from an agent’s neighbor table. Three weighing markers are later used to select an instance from the two. These markers are (1) disk usage, (2) available free memory, and (3) boot image availability at an OpenStack instance. However, the random selection has the following limitations:
• There is no guarantee that a resource is found, even though it might exist somewhere on the P2P system.
• It lacks the ability to process and satisfy a geographical location-based scheduler instruction. The randomness in the filtering pro-cess makes it challenging to apply a weighting based on geograph-ical location makers.
These shortcomings are motivations for our location-based search mechanism introduced in upcoming sections.
3.4
Summary
This chapter introduced P2P systems, their properties and the chal-lenges around P2P membership management. We also talk about CY-CLON, a membership management protocol for unstructured P2P net-works. This chapter also detailed on the proposed P2P OpenStack ar-chitecture in Section 3.3.2. Additionally, we also provide insight into existing workload scheduling strategies employed by the P2P Open-Stack architecture.
Chapter 4
Location-based P2P lookup
In this section, we describe our approach to solve the location-based resource lookup problem for a P2P OpenStack system. We also ex-pect that the solution can be used to solve other resource lookups in a generic P2P system. Keeping this in mind, we present the semantics of our solution in general terms.
The upcoming sections are organized as follows. Section 4.1 de-scribes an overall architecture of our solution. This is followed by Sec-tion 4.2which introduces and defines the key terms used in our solu-tion. Specific requirements of the P2P OpenStack system and how our approach can solve them are detailed in subsections that follow. Later, we define the query language processed by the system in Section4.3. Section4.4has the algorithm and implementation of NSBootstrap, our node bootstrap protocol. This is followed by the architecture and im-plementation of our search protocol NSSearch, in Section4.5.
4.1
Search architecture
Our P2P search architecture is described in Figure 4.1. This architec-ture is inspired from the spatial search system described by Uddin et al. in [21]. The search is designed to run on top of a decentralized P2P system. Users submit their queries on a node, termed search node in the system and expect results to their query retrieved on the same node.
As described in Section 3.3.2, the P2P plane of a P2P OpenStack system consists of agent nodes running on top of OpenStack instances. These agents form the P2P layer as described in Section3.3.2. The
Figure 4.1: Search architecture, figure inspired from [21]
tion proposed by this thesis is an indexing-based solution. Hence, for a node, it (1) creates an index, (2) maintains an index and (3) executes a search on top of this index.
The creation, maintenance, and traversal of this index are major contributions of this thesis work. This index is termed the finger table of a node throughout this report. Items in this table are termed fingers for a given node. This is analogous to the concept of neighbors in mem-bership management protocols listed in Section3.2. The algorithm ex-pects each node in the system to have a unique label associated with it. The semantics of this label is described in upcoming sections. The algorithm that initializes the finger table is denoted as NSBootstrap. NSBootstrap also triggers a finger table population and maintenance protocol, CYCLON. An overview of the lifetime of a node executing our solution is described in Fig 4.2. NSSearch is the location-based search algorithm which makes use of the finger table presented in this thesis. These algorithms are described in detail in upcoming sections.
CHAPTER 4. LOCATION-BASED P2P LOOKUP 17
Once a bootstrapping node attains a minimum set of location infor-mation of the system in its finger table, we term the node has attained a stable state. Defining this stable state for the system is also a contri-bution of this report.
4.2
Abstract Namespace Tree (ANT)
In this section, we present our label-based approach to solve the location-based lookup problem identified in this report.
Popular distributed systems like Cassandra recommends admin-istrators to use consistent naming across their replicas within data-centers [22]. These names/labels help replicas in routing decisions within the system. Cloud providers like Amazon EC2 also have avail-ability zones which are labels of its data-centers like eu-central-1, eu-north-2, etc.
We extend this labeling approach to hold more information to assist nodes in a P2P system to execute a location-based resource lookup. We propose a labeling scheme that is (1) hierarchical at the naming level (2) extends a common prefix, basePrefix, and (3) represents the phys-ical location of a node. The labeling scheme creates a tree rooted at
basePrefix for a P2P system termed ANT. The approach can be
rep-resented as an enhanced use of Routing Indexes described in Section
2.3.2.
Our solution is different from general tree-based solutions in the following ways: (1) physical nodes are not placed in a hierarchical or-der on the ANT and are present only at the leaf level, and (2) a node perceives its position on the ANT through its label. Hence, the notion of the tree is an abstract one and we term it as an Abstract
Names-pace Tree of the system. The abstract nature of the ANT provides a node the freedom from (1) storing extra state information of its related nodes (parent, peers, and children) and (2) maintaining these relations during churns, failures or exit.
In our approach, the naming across participating P2P nodes forms an ANT rooted at basePrefix. Peers are labeled uniquely extending this basePrefix with string tokens separated by a delimiter. For a node in the P2P OpenStack system, these string tokens can be a representation of their physical location.
is an example label extending a basePrefix root.. The label has information about the physical location of node1 in the P2P system.
Figure 4.3 depicts an ANT extending a basePrefix - root.. In this example, we assume that string tokens a, b, ...h denote the physical lo-cation of nodes 1..9. We will be using this representation instead of the actual geographical location of nodes throughout this thesis report.
Figure 4.3: An ANT formed out of consistent labeling extending basePrefix root. Physical P2P nodes are at the leaf level (shaded gray).
node4 in Figure4.3has a label: root.a.b.node4.
Additionally, we define the following properties of the label of a P2P node present in an ANT:
1. prefixLevels: For a given label, we identify prefixLevels as exten-sions of the basePrefix in its label. A P2P node root.a.b.node4 in Figure4.3 has the following prefixLevels: root., root.a., root.a.b.
CHAPTER 4. LOCATION-BASED P2P LOOKUP 19
Table 4.1: childPrefixes for all prefixLevels of root.a.b.node4 from the sample tree in Figure 4.3. The prefixLevel root.a.b. does not have a childPrefix as it does not have any non-leaf nodes on the ANT.
prefixLevel childPrefixes
root. root.a., root.d., root.e. root.a. root.a.b., root.a.c.
root.a.b. −
To summarize, some properties of the ANT which our algorithms utilize are:
1. Actual physical P2P nodes are present only at the leaf of the ANT. 2. basePrefix is a property of the P2P system. Nodes are assigned
labels extending basePrefix prior to bootstrap.
3. A physical node belongs to all its prefixLevels. A P2P node root.a.b.node4belongs to root., root.a., root.a.b. prefixLevels.
Why not use DNS for location-based lookups?
DNS is hierarchical, centralized and builds a tree rooted at its Top Level Domain (TLD) in the Domain Name Space. For example, a do-main www.example.com has a TLD of .com. A dodo-main name is an extension of its parent node in the Domain Name Space, separated by a dot. For our specific use case of the location-based resource lookup in P2P systems, we do not use the DNS system as it requires physical nodes to be arranged in a hierarchical fashion. Moreover, a leading principle of our P2P OpenStack architecture is to avoid any supern-odes or centralized components in the system [2].
4.2.1
Tracking the state of an ANT
Our solution architecture expects a node to build an index (finger ta-ble) to track the state of the ANT it is part of. We specify the following limited information a P2P node should keep track of:
Figure 4.4: A P2P node should keep track of at least one node in all childPrefixes of all its prefixLevels. In effect, node4 should keep track of at least one node on all branches of the ANT which includes a shaded token (along with its leaf sibling, node2).
identifier is also stored, which is not explicitly mentioned but assumed.
2. A finger table with a list of all its prefixLevels along with informa-tion of at least one node in every childPrefix for each of them. The actual number of nodes to keep track of is configurable, and its impact on our solution is detailed in Section4.4.3later.
3. Additionally, information of all of its leaf siblings on the ANT. A P2P node root.a.b.node4 in Figure4.3 should store infor-mation of root.a.b.node2 at its root.a.b. prefixLevel. This ensures that every node in the system belongs to at least one pre-fixLevel of some other node in the ANT.
As an example, Figure4.4describes the prefixLevels a node
root.a.b.node4should keep track of. These prefixLevels are shaded gray. The actual state information stored would be the network iden-tifier and label of a node in its corresponding prefixLevels.
CHAPTER 4. LOCATION-BASED P2P LOOKUP 21
Table 4.2: Description and properties of key terms used throughout this report.
Term Description Properties
basePrefix top level string token of the ANT
- All nodes belong to basePrefix - Example: root.
prefixLevel extension of basePrefix
- determines the size of finger table - Example: root., root.a., root.a.b.
childPrefix immediate non-leaf child of a prefixLevel on an ANT
- Example: root.a, root.d., root.e.are childPrefixes for root. prefixLevel
4.2.2
Use case: Geographical locations
The geographic location of a node can be used to generate logical la-bels for participating nodes. Classical address identifiers can also be used to label a node, which can assist later with location-based re-source lookups.
For example, a labeling scheme of the form root.continent.country.
state.city.street datacenter.nodeNamecan assist our search algorithm in efficiently locating a node at a given location. This is the exact use case addressed by this thesis for a P2P OpenStack system. A sample setup following this naming scheme is produced in Figure4.5.
4.3
The query language
In this section, we define the query language used by our search pro-tocol. We denote it in equation 4.1 below using BNF notations. In its raw form, namespace based labeling can support a query to find searchCount nodes matching a prefix searchPrefix in the system’s ANT.
query : pref ix = searchP ref ix ∧ count = searchCount (4.1) Notation4.1denotes an ∧ (AND) operator between two query pa-rameters: (1) prefix, and (2) count. The search returns searchCount re-sults for a given searchPrefix if found in the system. An example query processed our search mechanism is:
query : (pref ix = root.a.b.) ∧ (count = 5)
4.4
NSBootstrap: Node bootstrap protocol
We term the protocol that is responsible for initializing a finger table for a node NSBootstrap. The protocol also kick-starts the finger table population and maintenance protocol, CYCLON for a P2P node.Pre-conditions: A P2P node is configured with its label and a pointer to an introducer node in the system. The introducer has pointers to at least one other node which belongs to basePrefix level. Since all P2P nodes on the ANT belong to basePrefix, the introducer stores
point-ers to a random subset of P2P nodes in the system. Newly joined nodes are configured to update the introducer with their information on start-up.
CHAPTER 4. LOCATION-BASED P2P LOOKUP 23
1. Node n determines its prefixLevels L1, L2...LN and adds its in-formation to all these levels. This initializes its finger table. 2. Node n updates the introducer about it joining the system.
Si-multaneously, it requests for pointers to other nodes in
basePre-fix. The response is saved as fingers for the basePrebasePre-fix.
3. With CYCLON, node n exchanges its information with its
basePre-fix fingers growing the finger table for basePrebasePre-fix. As soon as it
finds a node matching its next prefixLevel L2, NSBootstrap
ad-vancesto that prefixLevel.
4. Node n continues exchanges at this prefixLevel, growing and ad-vancing through its finger table until all its prefixLevels have been traversed.
The exchanges mentioned in (3) and (4) steps above populate and maintain the finger table for a node. These exchanges are programmed to maintain information about all parts of the ANT, a P2P node is ex-pected to keep track of. The actual workflow of these exchanges is discussed later in this section.
Algorithm 1Pseudocode for a node n joining our ANT based system executing NSBootstrap protocol.
Input: introducer- pointer to the introducer node; myLabel - label of n.
Result: Finger table N populated; maintenance protocol CYLCON triggered for all prefixLevels of the node.
Data structures: 1: F :=finger table of n message types: 2: message(IPAddress, Label) NSBootstrap(introducer, myLabel): 3: prefixLevels ← getAllPrefixLevels(myLabel) 4: forprefix in prefixLevels do
5: F[prefix].append(myLabel) 6: end for 7: fingers ← introducer.RetreiveForPrefix(basePrefix) 8: F[basePrefix].append(fingers) 9: nextPrefix ← getNextPrefix(basePrefix) 10: PopulateAtLevel(basePrefix, nextPrefix) 11: return True
PopulateAtLevel( currentPrefix, nextPrefix): 12: if nextPrefix == myLabel then
13: return True 14: end if
15: StartCYCLONAtLevel(currentPrefix) // Finger table population. Main thread waits until a node matching nextPrefix shows up. 16: targetFinger ← F[currentPrefix].fingerMatching(nextPrefix) 17: F[nextPrefix].append(targetFinger) 18: currentPrefix := nextPrefix 19: nextPrefix ← getNextPrefix(currentPrefix) 20: PopulateAtLevel(currentPrefix, nextPrefix) 21: return True
As mentioned, NSBootstrap (1) initializes the finger table for a node, and (2) kick-starts the finger table population and maintenance pro-cess, CYCLON.
The algorithm is divided as follows. Lines 3-11 represent the pri-mary execution thread of NSBootstrap. An auxiliary function
algo-CHAPTER 4. LOCATION-BASED P2P LOOKUP 25
rithm stops its execution once it has traversed through all prefixLevels of a node. However, it triggers a background finger table population and maintenance process StartCYCLONAtLevel (line 15) for each pre-fixLevel. This background process terminates only once the node leaves the system (or in the event of a crash/failure). Moreover, Algorithm1
utilizes the utility functions described in Table4.3.
Table 4.3: Utility functions used in Algorithm1.
Line Utility Function Descripiton
3 getAllPrefixLevels(label) params: label
returns: prefixLevels for a label 7 node.RetrieveForPrefix(prefix)
params: prefix
returns: Fingers for a given prefix fetched from a node
9,
19 getNextPrefix(currentPrefix)
params: currentPrefix
returns: Next string token in label after currentPrefix
16 fingers.fingerMatching(prefix)
params: prefix
returns: Nodes matching prefix from a list of fingers
For a node n with a label myLabel, NSBootstrap starts by determin-ing its prefixLevels (line 3). A utility function is utilized for the same (line 3). For node root.a.b.node4. from our previous examples, this function would return the following prefixLevels root.,root.a., root.a.b..
Table 4.4: Finger table of node root.a.b.node4 from Figure4.3after ex-ecution of NSBootstrap lines 3-6.
prefixLevel Fingers
root. root.a.b.node4 root.a. root.a.b.node4 root.a.b. root.a.b.node4
NSBootstrap advances by fetching information about basePrefix from its introducer in line 7. The step also updates the introducer about node n joining the system. This process makes sure that the newly
joined node has information about at least one other node on the ANT. This step also ensures connectivity throughout the P2P system, which is explained later in this Chapter. The state of the finger table for a node root.a.b.node4 after this step is described in Table4.5. Table 4.5: Finger table of node root.a.b.node4 from Figure4.3after ex-ecution of NSBootstrap lines 7-8.
prefixLevel Fingers
root. root.a.b.node4, root.e.f.g.node6 root.a. root.a.b.node4 root.a.b. root.a.b.node4
Once the algorithm has received information about another node in basePrefix, it calls an auxiliary function PopulateFingersAtLevel on line 10. The function performs the following steps:
1. It triggers CYCLON, the finger table population and maintenance process through StartCYCLONAtLevel for each prefixLevel (line 15). The process executes in the background and is explained in upcoming sections.
CHAPTER 4. LOCATION-BASED P2P LOOKUP 27
3. It recursively advances through all prefixLevels of a node execut-ing steps mentioned in (1) and (2) above.
CYCLONpopulates the finger table of a P2P node with shuffling rounds (Section4.4.1). An example state of the finger table of node root.a.b.node4after several rounds of CYCLON shuffling for root. prefixLevel is given in Table: 4.6.
Table 4.6: Finger table of node root.a.b.node4 from Figure4.3after sev-eral CYCLON shuffling rounds at root. prefixLevel
. prefixLevel Fingers root. root.a.b.node4, root.e.f.g.node6, root.d.node1, root.a.c.node3 root.a. root.a.b.node4 root.a.b. root.a.b.node4
Eventually, the auxiliary function iterates through all prefixLevels of a node. An exit condition is met for the last prefixLevel on line 12, and NSBootstrap terminates. However, the background maintenance processes spawned by the auxiliary function continue to execute until the node leaves the system. This approach has the following benefits:
1. A failure, crash or removal event at a P2P node is recognized by the maintenance process. When such an event is recognized, the maintenance process can remove that failing node from its finger table.
2. Information about new nodes joining the P2P system are propa-gated through the system.
The following section describes the finger table population and maintenance process in detail.
by NSBootstrap is the original work by Voulgaris et al. [16]. The al-gorithm is described in Section3.2.1. CYCLON has the advantage of being inexpensive and robust (time to converge, no circular relations). Voulgaris et al. define a system parameter Shuffle Length for CYL-CON exchanges. This constant is the maximum number of elements a node P exchanges with another node Q in one round of CYCLON. Since NSBootstrap executes CYCLON at multiple levels, the algorithm requires the following enhancements:
1. An extension to run multiple instances of a CYCLON process per P2P node. For NSBootstrap, an instance of the CYCLON protocol is executed per prefixLevel.
2. Redefining the CYCLON Shuffle Length to make sure that a node will preserve atleast one pointer to all childPrefixes for a prefixLevel after a CYCLON round. This is achieved by always exchanging Shuffle Length number of fingers per childPrefix for a given prefixLevel in a CYCLON round. In a CYCLON exchange from P to Q, this ensures that all childPrefixes for a prefixLevel known to P are discovered by Q. Additionally, P does not lose all pointers to a childPrefix for a prefixLevel after a CYCLON round.
One round of CYCLON
The original work by Voulgaris et al., defines a CYCLON round [16] as a single CYCLON shuffle operation. An overview of what happens during a CYCLON round at a prefixLevel during NSBootstrap for a node P is explained below:
1. The age of all fingers in a given prefixLevel is incremented by 1. 2. P selects a peer Q with the highest age from its finger table row
for prefixLevel.
3. P exchanges information about Shuffle Length (if available) num-ber of its own fingers per childPrefix for the prefixLevel with Q. The selected fingers are modified as per original CYCLON rec-ommendations prior to the exchange.
4. P receives from Q information about Shuffle Length number of fingers per childPrefix for the same prefixLevel.
CHAPTER 4. LOCATION-BASED P2P LOOKUP 29
NSBootstrap search index and CYCLON finger tables
For a P2P node executing our solution, NSBootstrap creates a search index termed finger table. This index is later used by our distributed search algorithm to traverse the P2P network. NSBootstrap uses CY-CLON to populate and maintain this finger table. CYCY-CLON requires a finger table to execute its shuffling operation. The distinction between the search index built per P2P node by NSBootstrap and the actual finger tables maintained by CYCLON is represented in Table4.7.
For a P2P system executing NSBootstrap, CYCLON can be replaced with an indexing based approach for P2P search (Section: 2.3) that generates a search index similar to Table 4.7 (a). However, CYCLON is preferred in our solution as the P2P OpenStack architecture utilized the protocol for membership management [2].
Table 4.7: The search index built by NSBootstrap (a), and the actual fin-ger tables built and maintained by CYCLON (b-d) for root.a.b.node4 from Figure4.3.
(a) Search index built by NSBootstrap for node root.a.b.node4 (Figure4.3)
prefixLevel Fingers root. root.a.b.node4, root.e.f.g.node6, root.d.node1, root.a.c.node3 root.a. root.a.b.node4, root.a.b.node2, root.a.c.node3 root.a.b. root.a.b.node4 root.a.b.node2 (b) CYCLON finger table
for root. prefixLevel
Finger Age
root.a.b.node4 0 root.e.f.g.node6 2 root.d.node1 3 root.a.c.node3 4
(c) CYCLON finger table for root.a. prefixLevel
Finger Age
root.a.b.node4 1 root.a.b.node2 3 root.a.c.node3 2
(d) CYCLON finger table for root.a.b. prefixLevel
Finger Age
Connectivity of the network graph
The finger table of a P2P node stores pointers to other nodes in the P2P network. NSBootstrap uses CYCLON to build and maintain this finger table. Connectivity between P2P nodes is essential to ensure that a resource lookup through the finger tables returns a correct re-sult. Voulgaris et al. states that CYCLON guarantees the connectivity between P2P nodes in a fail-free environment [16] within a finite time. NSBootstrap inherits the property of connectivity from CYCLON.
However, there are edge cases for disconnection on the overlay net-work graph. A crash, failure or removal of a P2P node can result in a disconnection. We do not investigate these cases within the scope of this thesis.
Similar to most index-based solution, NSBootstrap eventually con-verges to a stable state. The properties of this stable state are described in Section4.4.2.
4.4.2
Stable state for NSBootstrap
A bootstrap algorithm is evaluated for the time and effort it takes for a new node joining a system to attain a stable state. We denote that a node executing NSBootstrap has acquired a stable state if and only if, it has the following in its finger table:
1. At least one entry (excluding itself) for each of its prefixLevels if it exists in the system.
2. At least one entry (excluding its own entry) for each childPrefix
for all its prefixLevels) if it exists in the system.
3. All entries of its leaf siblings in the ANT for its parent prefixLevel. The lifetime of a P2P node until it attains a NSBootstrap stable state is described in FigureA.1.
4.4.3
Optimize NSBootstrap resource utilization
CHAPTER 4. LOCATION-BASED P2P LOOKUP 31
impact on NSBootstrap resource utilization on a P2P node. We define the following parameters which can be configured to tune the execu-tion of NSBootstrap.
1. Maximum number of children nodes per childPrefix: This de-termines the maximum number of other node information stored per childPrefix for a prefixLevel. For NSBootstrap to function, a value of ≥ 1 is required to satisfy the stable state conditions (Sec-tion 4.4.2). A higher value for this configuration can speed up lookups as more data is available, but can increase the state stored per node.
2. CYCLON Shuffle Length: Determines the maximum number of fingers exchanged per CYCLON round at a prefixLevel per child-Prefix. For a node P executing NSBootstrap, a Shuffle Length of 1 would exchange one entry per childPrefix at a prefixLevel with a node Q in a single CYCLON round. A value ≥ 1 is required for this parameter.
3. CYCLON round intervals: The original CYCLON algorithm does not mandate the interval between successive shuffling rounds [16]. Reducing this value can speed up NSBootstrap. However, this can increase the number of requests per unit time in the sys-tem affecting CPU, network usage and eventually performance. This parameter should be tuned considering the volatility of nodes in the P2P network.
4.4.4
Factors affecting NSBootstrap convergence time
Section 4.4.2lists the conditions for a node executing NSBootstrap to attain a NSBootstrap stable state. For a P2P system, our solution con-verges after all nodes in the system have attained a NSBootstrap sta-ble state. Since a CYCLON round consumes time and computational power, we identify the following factors that can affect the time and effort it takes for a system to converge through NSBootstrap.information with peers (or siblings) on its branch on the ANT can reduce the number of CYCLON rounds required to attain a stable state. In contrast, exchanging information with nodes in a different branch on the ANT increases the number of CYCLON rounds to attain the same result.
2. Being introduced to a node in own branch: A P2P node is intro-duced with information of any other random node in the system. However, introducing a node P to an existing node Q on its own branch on the ANT can reduce the number of CYCLON rounds P would take to attain a stable state. The contrary results in more number of CYCLON rounds to attain the same result.
These factors are used to explain and form assumptions later in Chapter5.
4.5
NSSearch: The lookup protocol
In this section, we introduce our lookup protocol, NSSearch. The pro-tocol is dependent on the NSBootstrap algorithm and is a major con-tribution of this thesis. The protocol assumes that (1) every node in the P2P system has initialized NSBootstrap, and (2) may or may not have achieved the stable state defined in Section4.4.2.
The query language processed by the search system is described in Section 4.3. An example query processed by NSSearch is to find
searchCount nodes in a searchPrefix prefix. NSSearch utilizes the
fin-ger table built with NSBootstrap as an index to navigate through the P2P system systematically. Additionally, we term the node executing a search as the search node. An overview of the NSSearch algorithm to find searchCount nodes in searchPrefix as executed by a search node P is described below:
1. P checks if searchCount nodes exists for searchPrefix in its finger table F. If yes, return these searchCount nodes as results. If not, the protocol advances through the next step.
2. P finds the longest prefix match between its label and searchPrefix. Fingers for this prefix are pulled from the finger table F. These fingers are further queried for nodes that match searchPrefix. 3. The algorithm further advances in an iterative or a recursive
CHAPTER 4. LOCATION-BASED P2P LOOKUP 33
For a search node P with label myLabel, the lookup in step 2 above translates to a string comparison between searchPrefix and myLabel. For example, assume root.a.b.node4 from Figure4.3as a search node, processing a query of the following form via NSSearch:
query : (pref ix = root.e.f.h.) ∧ (count = 2)
A longest prefix match between root.a.b.node4 and root.e.f.h. will result in:
longestM atch = longestP ref ixM atch(root.e.f.h., root.a.b.node4)
=root.
For node P, NSSearch now proceeds to figure out the next finger(s) to contact. NSBootstrap makes sure that a node would store informa-tion of at least one extra node per each of its childPrefixes for all its prefixLevels (Section 4.4.2). NSSearch leverages on this knowledge to query a node’s finger table to fetch other fingers matching the next childPrefix after longestMatch. In the above example, this would be:
childSearchP ref ix = (root.e.f.h.).getAf terP ref ix(root.)
=root.e.
For node P, NSSearch now has the knowledge of (1) longestMatch, the prefixLevel it has in its finger table F and (2) childSearchPrefix, the prefix which should match one or more of its longestMatch prefixLevel fingers. NSSearch identifies these fingers which could possibly an-swer a search query as possible searchTargets. In the above example of root.a.b.node4, this would be:
searchT argets = F [root.].match(root.e.)
=root.e.f.g.node6
4.5.1
Iterative traversal of an ANT by NSSearch
As the name suggests, a search node executing NSSearch is responsible for (1) finding the right nodes to query, (2) query these nodes, and (3) report the results back to the user. Algorithm2describes the pseudo-code for a node executing NSSearch in an iterative fashion.
Algorithm 2 Iterative NSSearch: Pseudocode for a node n receiving and executing NSSearch in an iterative fashion
Input: A query for count number of nodes in searchPrefix
Result: List with count number of nodes in searchPrefix (if found)
Data Structures:
1: F :=finger table of n;
NSSearch(searchPrefix, count)
2: ifF[searchPrefix].length ≥ count then 3: return F[searchPrefix].get(count) 4: end if
5: longestMatch← getLongestPrefixMatch(myLabel, searchPrefix) 6: childSearchPrefix ← getAfterPrefix(longestMatch, searchPrefix) 7: searchTargets ← F[longestMatch].match(childSearchPrefix) 8: results = NSSearchIterative(searchPrefix, count, searchTargets) 9: return results
NSSearchIterative( searchPrefix, count, searchTargets) 10: results = []
11: fortarget in searchTargets do 12: ifresults.length ≥ count then 13: break
14: end if
15: FOUND, newResults ← target.FindForPrefix(searchPrefix) 16: ifFOUND == TRUE then
17: results.append(newResults) 18: end if
19: searchTargets.append(newResults) 20: end for
21: return results
CHAPTER 4. LOCATION-BASED P2P LOOKUP 35
Algorithm2is organized as follows. The main execution thread of the program is described in lines 2-9. NSSearch starts by checking if the answer for a query exists in its own finger F table in lines 2-4. If not, it proceeds to figure out further searchTargets using an auxiliary func-tion NSSearchIterative in lines 10-21. Moreover, Algorithm 2utilizes the utility functions described in Table4.8.
Table 4.8: Utility functions used in NSSearch iterative implementation (Algorithm2).
Line Utility Function Descripiton
5 getLongestPrefixMatch( label, searchPrefix)
params: label, searchPrefix
returns: longest string match between label and searchPrefix
6 getAfterPrefix(
longestMatch, searchPrefix )
params: longestMatch, searchPrefix returns: next string token after longestMatch in searchPrefix 13 node.FindForPrefix(
searchPrefix)
params: searchPrefix returns:
tuple of (True, results) if results found else tuple of (False, further search targets) The auxiliary function NSSearchIterative (lines 10-21) iterates through
possible searchTargets to retrieve count number of results for the query. Once the algorithm has iterated through all searchTargets, this result list is returned as a response to the search query in line 9.
4.6
Summary
In this chapter, we have introduced and detailed the core contribu-tions of this thesis. We started by detailing the overall architecture of our proposed P2P search solution, followed by the concept of
Chapter 5
Evaluations and discussions
In this chapter, we detail the evaluation tests we did as part of this thesis work for NSBootstrap and NSSearch. Since we are introducing these two algorithms in this thesis, we define:
• Validation: We validate whether the algorithm delivers its ex-pected results.
• Evaluation: We evaluate the results of the algorithm in varying environments. We also evaluate the resource utilization (CPU usage) of the algorithm on P2P nodes.
5.1
Test Setup
In this section, we detail the testbed and mechanics used while vali-dating and evaluating our solutions.
All tests are run on a setup of N P2P nodes, with N varying from 108 to 3000. Each P2P node is deployed as a single Python process running on an HTTP port. To deal with test-bed limitations, we group multiple such processes on a single Virtual Machine (VM) and run the VMs in parallel. The VMs remain in a single logical network and each have a configuration of 8 Virtual CPU cores and 16 GB of RAM. Also, we deploy at the most 125 of such processes per VM to stay below 100% CPU usage (to avoid unrealistic results).
From an implementation perspective, the nodes start to execute NSBootstrap as soon as they receive their unique label from a script. Unlike a real-world setup, labels are assigned dynamically during the experiment. The Python processes we implemented are analogous to
agent processes running on top of an OpenStack installation in a P2P OpenStack system (Section3.3).
Moreover, P2P nodes represent the leaf level nodes of the system’s ANT for every configuration.
All tests are repeated 10 times and we define 3 parameters which are varied, one after the other. These 3 parameters define the total number of leaf nodes in the system and the shape/size of the system ANT formed. These 3 parameters are:
1. Depth (D): The depth of the ANT excluding its leaf nodes. An in-crease of this number has an exponential effect on the total num-ber of leaf nodes in the system.
2. K: The number of children every non-leaf vertex has on an ANT. This defines the expansion of the ANT or its width. An increase of this number has a polynomial effect on the total number of leaf nodes in the system.
3. Leaves (L): The number of leaf physical nodes on any branch on the ANT.
The total number of leaf P2P nodes in the system is hence deter-mined by the parameters D, K, L and is denoted as N.
Moreover, the following configuration parameters were used for running our evaluation tests:
1. Any new P2P node P is introduced to the system with the infor-mation of one additional node Q (P 6= Q) by a static introducer within the system. This mandates that the node P has to pop-ulate its finger table all the way from a single pointer (to Q) to eventually attain a NSBootstrap stable state (Section4.4.2). 2. For each prefixLevel in the system, a node P stores information
CHAPTER 5. EVALUATIONS AND DISCUSSIONS 39
(a) Initial, D=2, K=2, L=1
configura-tion. N=4 (b) Varying L, D=2, K=2, L=2 configu-ration. N=8
(c) Varying K, D=2, K=3, L=1 configuration. N=8
(d) Varying D, D=3, K=2, L=1 configuration. N=8
Figure 5.1: Sample ANT state for various configurations of D, K and L 3. The CYCLON Shuffle Length is set by default to 1. Refer to Sec-tion4.4.3for details on this parameter. The parameter is set to the lowest possible value it can take to stress-test the bootstrapping process. Moreover, a node is considered to have attained a stable state if and only if it satisfies all conditions for our NSBootstrap stable state (Section4.4.2)
5.2
NSBootstrap
NSBootstrap is a contribution of this thesis work and defines the boot-strapping phase of a P2P node. For a new node joining a system, the bootstrap phase is considered complete when it attains all of the NSBootstrap stable state requirements (Section4.4.2). Our strategy to evaluate NSBootstrap is described below:
1. Start N P2P nodes at the same time and assign all of them a unique label.
2. Monitor CPU usage and the number of CYCLON rounds required by all (N) nodes in the system until all nodes attain a NSBoot-strap stable state (Section4.4.2)).
3. Start the N + 1thP2P node and assign it a unique label.
4. Monitor CPU usage and the number of CYCLON rounds required by the N + 1thnode until it attains a NSBootstrap stable state.
5.2.1
Metrics measured
The metrics measured are:
1. CPU and memory usage by a node executing NSBootstrap until it attains a stable state.
2. The total number of CYCLON rounds executed at all prefixLevels by a node executing NSBootstrap until it attains a stable state. A CYCLON round in our setup is defined in Section4.4.1.
5.2.2
Validation
From the evaluation experiments, we see that all P2P nodes in the sys-tem attained a NSBootstrap stable state after a finite number of CY-CLON rounds. This validates our NSBootstrap protocol.
5.2.3
Evaluation
CHAPTER 5. EVALUATIONS AND DISCUSSIONS 41
(a) Varying D, with constant K and L
D K L Nodes Repeat 3 3 4 108 x10 4 3 4 324 x10 5 3 4 972 x10 6 3 4 2916 x10 Total runs 400
(b) Varying K, with constant D and L
D K L Nodes Repeat 3 3 4 108 x10 3 4 4 256 x10 3 5 4 500 x10 3 7 4 1372 x10 3 9 4 2916 x10 Total runs 500
(c) Varying L, with constant K and D
D K L Nodes Repeat 3 5 1 125 x10 3 5 2 250 x10 3 5 4 500 x10 3 5 8 1000 x10 3 5 24 3000 x10 Total runs 500
Table 5.1: Various configurations of D, K and L used for our experi-ments to evaluate NSBootstrap
A point on the resultant plot is the average measurement of 10 runs. The upper bound and lower bound of the error bar represent the max-imum and minmax-imum values obtained respectively.
5.2.4
Results and discussion
Varying D experiments
The results of varying D experiments are plotted in Figure5.2.
Observations
1. We see a linearly increasing line for the average number of CY-CLON rounds as Depth (D) increases.
(a) Average CYCLON rounds for N nodes in the system. The total number of nodes (N) is given inside parenthesis for the resulting configuration on the horizontal axis. A point in the plot is the average of N*10 measurements.
(b) Average CYCLON rounds for an N + 1thnode joining the system after N nodes have attained a stable state as per Section4.4.2.
CHAPTER 5. EVALUATIONS AND DISCUSSIONS 43
3. For a new P2P node N + 1 joining the system, we see that the range of measurements around the average CYCLON rounds is negligible.
Discussion
1. As expected, the total average number of CYCLON rounds in-crease linearly with depth D of the ANT. As D inin-creases for a P2P node on the ANT, it has to (1) keep track of more prefixLevels, and (2) hence results in more average CYCLON rounds executed per P2P node. This is observed for both an N + 1th joining the system as well.
2. We observe a varying range of measurements around the av-erage number of CYCLON rounds in Figure 5.2(a). This is ex-pected due to (1) the randomness in CYCLON protocol, and (2) the chance for a new P2P node to get introduced to another node in its own branch on the ANT. The latter can speed up NSBoot-strap and reduce the total number of CYCLON rounds required. 3. For a N + 1thP2P node joining the system, attaining a stable state is a predictable process. From Figure5.2(b), one can deduce that the total number of iterations required to attain a stable state for a node is directly proportional to the depth D of its label. 4. For a new N + 1th P2P node, the average number of total
CY-CLON rounds required is always the depth (D) of its label added to a fixed value (2 for this experiment (Figure 5.2 (b)). This is explained below with an example.
Assume a new P2P node root.c.c.c.108 joining a P2P system with an ANT of K=3, L=4 and D=3 configuration. It is presumed that all other 107 nodes in the system have attained a NSBootstrap state already as per Section4.4.2. Lets assume that the static introducer node in the setup provides root.c.c.c.108 information about another node root.b.a.a.54. The following steps are responsible for its 5 total CYCLON rounds:
2. Round=2, exchange and add information of root. from
root.b.a.a.54. This fetches information of at least 1 node in root.c.prefix. Lets assume this to be root.c.a.a.node53. 3. Round=3, exchange and add information of root.c. from
root.c.a.a.node50. This fetches information of at least 1
node in root.c.c. prefix. Lets assume this to be root.c.c.a.node52. 4. Round=4, exchange and add information of root.c.c. from
root.c.c.a.node52. This fetches information of at least 1 node in
root.c.c.c.prefix. Lets assume this to be root.c.c.c.node51. 5. Round=5, exchange and add information of root.c.c.c. from
root.c.c.c.node51. This fetches information of all nodes in root.c.c.c.prefix. Lets assume this to be [root.c.c.c.node50, root.c.c.c.node49..] .
6. Node root.b.a.a.54 has achieved a stable state as per Section
4.4.2.
Varying K experiments
The results of varying K experiments are plotted in Figure5.3.
Observations
1. We do not observe any direct relationship between the measured value for the average number of CYCLON rounds and K.
2. Comparing Figure 5.3(a) and 5.2(a), we observe that the maxi-mum average number of total CYCLON rounds required by N nodes in the system are significantly lower in the former plot for similar values of N. For example, with N=2916, a D=6, K=3 con-figuration requires about 250 CYCLON rounds on average per P2P node to attain a NSBootstrap stable state. However, for the same N, a D=3, K=9 configuration requires only about 50 CY-CLON rounds on average per P2P node to attain a NSBootstrap stable state.
3. For a new N + 1thP2P node joining a system, increasing K seems not to have an effect on its number of CYCLON rounds as per
