Adaptive management for networked systems

(1)

Adaptive Management for Networked Systems

ALBERTO GONZALEZ PRIETO

Licentiate Thesis Stockholm, Sweden, 2006

(2)

School of Electrical Engineering KTH, Stockholm, Sweden Akademisk avhandling som med tillstånd av Kungl Tekniska högskolan framläg- ges till offentlig granskning för avläggande av teknologie licentiatexamen onsdag den 7 juni 2006 i KTH, Stockholm.

(3)

Abstract

As networked systems grow in size and dynamicity, management systems must be- come adaptive to changing networking conditions. The focus of the work presented in this thesis is on developing engineering principles for adaptive management systems.

We investigate three problems in the context of adaptive management for networked systems. First, we address the control of the performance of an SMS system.

We present a design for policy-based performance management of such systems. The design takes as input the operator's performance goals, which are expressed as policies that can be adjusted at run-time. The system attempts to achieve the given goals by periodically solving an optimization problem that takes as input the policies and traffic statistics and computes a new configuration. We have evaluated the design through extensive simulations in various scenarios and compared it with an ideal system. A prototype has been developed on a commercial SMS platform, which proves the validity of our design.

Second, we address the problem of decentralized continuous monitoring of network state variables with configurable accuracy. Network state variables are computed from device counters using aggregation functions, such as SUM, AVERAGE and MAX. We present A-GAP, a protocol that aims at minimizing the management overhead for a configurable average error of the estimation of the global aggregate.

The protocol follows the push approach to monitoring and uses the concept of incremental aggregation on a self-stabilizing spanning tree. A-GAP is decentralized and asynchronous to achieve robustness and scalability. We evaluate the protocol through simulation in several scenarios. The results show that we can effectively control the fundamental trade-off in monitoring between accuracy and overhead.

Third, we aim at improving the performance of the policy distribution task: the mechanism that provides the right policies at the right locations in the network when they are needed. Policy distribution is a key aspect for developing policy-based systems that scale, which is a must for dynamic scenarios. We present a scalable framework for policy distribution. The framework is based on aggregating the addresses of the policies and applying multipoint communication techniques. We show the validity of the framework in a case study.

(4)

Acknowledgments

First, I would like to thank my advisor Rolf Stadler, for his support, for the fruitful discussions, and for introducing me into the world of scientific research. I would also like to thanks all the people at LCN for the stimulating atmosphere, which made it possible for me to focus my efforts on research.

Additionally, I would like to express my gratitude to the organizations and entities that have founded my research: VINNOVA (the Swedish Governmental Agency for Innovation Systems), the Graduate School in Telecommunications at KTH, and the Ambient Networks - IST project.

I would like to thank my friends for their help. I would like to express my special thanks to my family for their continuous support and to Elena for her understanding, support and patience.

(5)

Curriculum Vitae

Alberto Gonzalez Prieto is a PhD candidate at the Royal Institute of Technology, Stockholm, Sweden. His research interests include scalability and self-configuration in network management.

He received his M.Sc. in Electrical Engineering from the Universidad Politecnica de Cataluña, Barcelona, Spain in 2002.

He was an intern at NEC Network Laboratories (Germany) in 2001,working on his Master Thesis.

(6)

(7)

1. Introduction

The work presented in this thesis is structured into three research lines: policy- based performance management for SMS systems, distributed real-time monitoring with accuracy objectives, and scalable policy distribution. In all of them, the focus is on adaptive management. Each line of work presents a management system that aims at achieving a goal autonomously, adapting to network state changes and external events.

1.1 Policy-based Performance Management for SMS Systems

The aim of this line of work is to develop a design that permits a node in a networked environment to achieve a goal in an autonomous fashion. Given a goal, the node must decide and execute the appropriate actions (considering its capabilities) that lead to the achievement of its goal.

The design must be adaptive, controllable, and have low and controllable computational overhead. Adaptability means that the design must be able to react to changes in networking conditions so that, despite these changes, it achieves its goal.

Control is a must in every management system. While the node must be autonomous in its decision-making process, the administrator must have the capacity to guide or control its behavior. The administrator must be allowed to set limits to the node behavior. This control can be expressed as a set of forbidden actions and/or forbidden states.

In order to be practically feasible, the computational cost of the design must be low and controllable. Low cost is a must for real-time management. It is crucial for achieving timely adaptability. Controllability permits vendors to bound the processing resources required by management tasks running on their devices.

Our research in single-node goal achievement has been done in the context of SMS messaging. Figure 1.1 presents the network architecture for SMS deployment in the GSM context. Specifically, we have considered the management of a key block of the SMS architecture: the SMS gateway, which is responsible for interconnecting different networks.

We present a design that enables the gateway to achieve the performance goals specified by the operator.

The operator can control the node behavior by specifying a list of policies that de- fine the set of forbidden states. This set establishes the range of values that must be avoided for the performance metrics of interest, such as delay. The actions taken by the gateway can not produce a degradation below a given threshold.

In our design, the determination of the actions that lead to achieve the goal is based on solving periodically an optimization problem.

(10)

We have shown through simulation that the design is able to achieve its goal while behaving according to the limits specified by the administrator. Our simulations also show that the system effectively adapts to network condition changes. The low computational cost requirement is also met: the decision-making process requires just a few milliseconds in a current desktop CPU.

In addition, as a proof-of-concept, a prototype has been developed on a commercial SMS platform.

1.2 Distributed Real-time Monitoring with Accuracy Objectives

In this line of work, we pursue a design that enables a large set of nodes in a networked environment to achieve a goal through cooperation. Given a goal, the nodes must determine and execute the appropriate actions (considering their capabilities) that lead them to the achievement of their goal.

As in the previous line of work, the design must be adaptive, controllable, and have low and controllable computational overhead. However, achieving these properties presents significant differences with respect to the single-node case. In this regard, a key aspect is the coordination among nodes. If the nodes do not coordinate their actions appropriately, it might be impossible to achieve the goal. For instance, two un- coordinated nodes may perform opposed actions that cancel each other. Another example is an erratic node jeopardizing the goal achievement.

SMSC

SMSG

Operator A Network Operator B Network

The Internet

SMS Application

HLR HLR SMSC

Figure 1.1: Functional Architecture for SMS. This work focuses on the SMS Gateway Performance Management.

(11)

In addition, controlling the behavior of a set of nodes is more intricate in the multi- node case. Controlling each node individually might not produce the desired behavior.

The actions of all nodes must be considered as a whole.

The computational overhead has an additional dimension when considering a set of nodes: its distribution among the nodes. In the general case, it is undesirable having a subset of the nodes heavily loaded, while others are almost idle.

Our research has been centered in the context of continuous monitoring of large- scale networks. Specifically, on the monitoring of network-wide metrics computed from device counters using aggregation functions, such as SUM, AVERAGE and MAX. Examples of such metrics include the total number of VoIP flows and the maximum link utilization in a network domain.

We present a decentralized and asynchronous protocol (A-GAP) that enables the nodes to achieve performance goals regarding the monitoring task. The goal of the system is to minimize the generated overhead for a configurable accuracy of the estimation. A-GAP addresses this system-wide goal by mapping it onto a local goal for each node. Each node attempts to minimize its processing load for a local accuracy constraint. The local accuracy constraints are determined considering the global accuracy constraint.

The protocol computes aggregates by (i) creating and maintaining a self-stabilizing spanning tree and (ii) incrementally aggregating the variables (see figure 1.2). AGAP is push-based: changes in monitored variables are sent towards the management sta-

4

22

Aggregating Node

Leaf Node Global

Aggregate Partial

Aggregate Local variable

Root

Physical Node Management Station

1

7 2

5 3

10

7 2

5 3

12

Figure 1.2: Example of aggregation tree on the management overlay. The ag- gregation function is SUM.

(12)

tion along the aggregation tree. When the reported changes are small, these messages can be dropped by (dynamically adaptable) filters installed in the tree nodes.

The configuration of the filters controls the performance of the monitoring task in terms of estimation accuracy and overhead. We have developed a stochastic model for the monitoring process that relates filter configuration and monitoring performance.

A-GAP pursues the system’s goal by determining the appropriate set of filters.

In our design, each node determines the actions to take (filter configurations) by solving periodically an optimization problem, which takes as arguments its own state and that of its neighbors. This computation is based on our stochastic model.

We have evaluated the protocol through simulation. Our experiments show that we can effectively control the fundamental trade-off between achieving high monitoring accuracy and maintaining low management overhead. High quality comes at the cost of a higher overhead and, similarly, a low estimation quality can be achieved with low overhead.

We have benchmarked our protocol against a rate-control scheme that controls the management overhead by limiting the update rates. For the same accuracy objective, our protocol incurs in a lower overhead than this approach. That is, in terms of goal achievement, A-GAP outperforms this alternative.

A-GAP enables performance prediction at run-time. Based on our stochastic model, a manager can be provided with an estimation of the expected load on all nodes and the distribution of the estimation error at the root node for a given accuracy objective. This is potentially significant in real scenarios. For instance, a manager could avoid overloading the management system by loosening the error objective.

1.3 Scalable Policy Distribution

In this line of work, the focus is also in a large set of nodes in a networked environment that must achieve a goal through cooperation. In this case, though, the type of goal is different. The system’s goal is to reach and stay in a given configuration state. The system must be able to keep (or return to) this state in case of networking condition changes.

Our research has focused on the policy-based management field. We consider scenarios where each node in the system is assigned a set of policies to govern its decision-making process. The system’s objective is that all the nodes have the right policies available for their evaluation when needed. This objective is challenged by changes in the policy set or in the node set.

A key requirement for policy distribution is timeliness. The system must be able to achieve the specified state rapidly. It must make available the right policies at the right locations with a low latency, even for dynamic and large-scale scenarios. Failing to meet timeliness may lead to inconsistent states where some nodes do take decisions based on an incorrect set of policies. In other words, long response time can make the system behave erratically.

Controlling the performance (generated overhead) of the policy distribution task is crucial for achieving timeliness. Node congestion would result in long delays in the policy distribution. Therefore, the system must be scalable in terms of generated management traffic.

(13)

We have proposed a scalable framework for policy distribution. It is based on aggregating the addresses of the policies and applying multipoint communication techniques. The aggregation is based on grouping the managed elements by the role they play in the network and distributing policies that apply to all the elements in a group.

Our proposal scales with the unicast routing algorithm deployed in the network.

The validity of the approach has been shown by applying it to a realistic scenario.

(14)

(15)

2. Related Research in Monitoring with Quality Objectives

We intend to focus our future research on extending the work on distributed real- time monitoring with accuracy objectives. In this section, we relate our work on this area to other works in the fields of wireless sensor networks and queries on data streams.

First, we classify monitoring queries in order to position our work with respect to that in the literature. Section 2.2 discusses different types of aggregation graphs. In section 2.3, we present recent results on a fundamental trade-off in monitoring: quality of the estimation versus the required monitoring resources.

2.1 Types of Monitoring Queries

We classify monitoring queries on aggregates into 3 categories:

• 1-time queries refer to a state snapshot. They request an estimation of the aggregate at a specific time.

• N-time queries request an estimation of the aggregate at discrete times. N- time queries are usually periodic.

• Continuous queries request a continuous estimation of the aggregate.

Most of the existing works on queries on aggregates focus on n-time queries. They are commonly realized as periodic and independent 1-time queries [42, 52, 33, 35, 36, 54]. In contrast, our work addresses continuous queries, like [48, 47, 49].

Note that this query classification is not universal in the literature. When some au- thors talk about continuous queries, they are referring to what we here call n-time queries (e.g., [42, 21]).

2.2 Aggregation Graphs

A key concept in distributed computation of aggregates is the aggregation graph. It determines how partial aggregates flow towards the monitoring station(s) and how they are aggregated along that path.

(16)

2.2.1 Trees

The most common aggregation graph is the tree [33, 42, 36, 32, 59, 54, 52] and several algorithms have been proposed for their creation and maintenance [50, 23, 19]. A key property of trees is that they guarantee that any two vertices are connected by exactly one path. This enables straightforward algorithms for aggregate computation that guarantee that each local variable is only accounted once. We discuss the importance of this property below.

The drawback of trees as aggregation graphs is that they are particularly sensitive to losses [46]. For instance, consider a 1-time SUM query. If a partial aggregate sent by a node with a large number of children is lost, the aggregate estimation can be se- verely affected. Therefore, trees are not well-suited for lossy environments.

2.2.2 Multi-path Graphs

To overcome the sensibility of trees to losses, [46, 13] propose the use of multipath aggregation graphs instead. These approaches aim at reducing the effects of lossy links by providing path diversity: local variables flow towards the monitoring station(s) following multiple paths.

In contrast to trees, multi-path graphs potentially introduce double accounting: a local variable is considered more than once in the aggregate computation. The effects of double accounting depend on the considered aggregate function. [42] classifies aggregates into two categories. First, duplicate-insensitive aggregates. The computation of these aggregates is not affected by double accounting. In other words, their computation is an idempotent operation. Some examples are MAX and MIN. Second, duplicate sensitive aggregates, like SUM. For this type of aggregates, duplicate accounting distorts the estimation.

The computation of duplicate-sensitive aggregates using multi-path graphs brings the challenge of avoiding double accounting while allowing incremental aggregation in the aggregation graph. To achieve this, [46, 13] make use of the probabilistic counting algorithm by Flajolet and Martin [28]. This algorithm provides an estimation of the count aggregate for 1-time queries. This work was extended to estimate sums by [13]. The key of the algorithm is that it keeps, for each node, a synopsis of its partial aggregate that probabilistically avoids double accounting. While double accounting is not fully avoided; the algorithm provides guarantees on the bias and standard deviation of the aggregate estimation. The quality of the estimation is controlled by the synopsis size. We will discuss this further in the next section. A drawback of this algorithm is that it requires families of hash functions with very strong independence properties [7] and it is vulnerable to adversarial local variables combinations [13].

A-GAP uses an aggregation tree instead of a multi-path approach for two main rea- sons. First, trees require nodes in the overlay to process fewer messages than multi- path graphs. A-GAP aims at minimizing the number of messages processed by a node; therefore, trees are preferred. Second, we consider scenarios with limited losses, where [46] and [13] perform worse than tree-based approaches in terms of aggregate

(17)

accuracy. In our scenarios, ISP networks, losses are not common and when they oc- cur, level 2 or level 4 mechanisms can mitigate their impact by retransmitting lost messages. In contrast, in wireless sensor networks, (those targeted by [46, 13]) lossy links are common and such mechanisms are unaffordable. Therefore, while multi-path approaches are attractive for wireless sensor networks, they are not for scenarios with a reliable network layer.

2.2.3 Effects of the Tree Topology in the Performance

The topology of the aggregation tree has a significant impact on the performance of the aggregation. Tree characteristics like its depth and the nodes’ degrees influence the number of messages processed by each node or the latency for presenting an aggregate estimation.

Despite the relevance of the tree topology, few works have investigated it. One of them is [35], which studies the communication costs of 1-time queries as a function of the relative location of data sources and the monitoring station. [54] analyzes how placing the monitoring station in different locations impacts the performance.

[52] introduces a new aspect in tree topology creation: the consideration of the query semantics. It proposes an algorithm that takes them into consideration to reduce the generated traffic for 1-time queries having (sql-like) "group by" clauses.

A-GAP uses GAP [19] to create and maintain an aggregation tree. Given an overlay, GAP creates a tree that minimizes the distance of each node to the root. There- fore, the choice of the overlay influences the performance of A-GAP. Currently, for the sake of simplicity, the overlay we choose has the same topology as the underlying physical network. How to improve the performance of A-GAP by selecting different overlays is an open issue.

2.3 The Trade-off between Quality of Estimation and Monitoring Resources Monitoring presents a fundamental trade-off: quality of the estimation (QoE) versus required resources. Achieving high quality estimations requires significant amounts of resources, and limited resources possibly mean low quality estimations.

The quality of an estimation has two aspects. First, the accuracy: how close the estimation is to the actual value. Second, the latency: how long it takes to present an estimation of the current aggregate to the monitoring station(s).

In the literature, we can find different instantiations of this trade-off. Next, we classify them into three categories: latency vs overall traffic, maximum error vs overall traffic, and accuracy vs storage requirements.

2.3.1 Latency vs Overall Traffic

Several papers on monitoring wireless sensor networks (WSN) [33, 36, 35, 42]

discuss a trade-off between latency and overall traffic or overall dissipated energy for 1-time and n-time queries.

(18)

In WSN, the overall traffic and the overall dissipated energy are closely related:

radio communications dominate energy consumption in wireless sensors [42]. There- fore, there is an almost linear dependency between overall traffic and overall dissipated energy. In the literature, some authors consider the overall energy as the parameter to minimize, while others consider the number of transmitted bits or messages. We group all of them under the same category.

The importance of these two metrics, in WSN scenarios, comes from the fact that the lifetime of a WSN, a key aspect of these networks, depends on them. A WSN lifetime is limited by the batteries of its sensors. Note that battery replacement is not an option in most realistic scenarios. Therefore, the maximization of the network lifetime is achieved by minimizing the energy consumption, i.e., the number of messages transmitted.

In WSN, it is common to use local synchronous in-network aggregation algorithms in order to reduce the generated traffic for answering 1-time and n-times queries. In such algorithms, each node reports its partial aggregate to its parent in the aggregation tree once it has received the reports from its children with their partial aggregates.

This creates a wave of updates towards the monitoring station(s). This enables the creation of scalable approaches in terms of generated traffic: the number of messages sent by each node in the distributed system is limited to one per 1-time queries [42, 35, 36] (1 message per period for n-times queries).

The drawback of such synchronous algorithms is that they introduce latencies in the delivery of reports. Since a node can only send its partial aggregate once it has received the partial aggregates from all of its children, data may be held for some time at some intermediate node in the aggregation tree. In the worst-case, the incurred latency is proportional to the tree depth [35, 36].

A-GAP, in contrast, does not hold reports. They may be filtered out, based on their value, though. This means that significant changes in the monitored variables are de- livered promptly to the monitoring station. In contrast, proposed synchronous approaches do not take into consideration the contents of the report.

Another difference between our work and these ones is that A-GAP aims at minimizing the maximum processing load over all nodes rather than the overall traffic.

This difference comes from the fact that we do not consider WSN scenarios. As discussed above, in WSN, the metric of interest is the overall traffic (or overall dissipated energy) because it is associated to the network lifetime. In contrast, for other networked scenarios, such as fixed networks or WLANs, the challenge is to avoid that the monitoring overhead overloads the nodes, especially in large-scale and dynamic networks.

Note, that none of the works discussed in this section aims at controlling the accuracy of the estimation.

2.3.2 Maximum Error vs Overall Traffic

Another instantiation of the QoE-vs-resources trade-off studied in the context of WSN is the maximum error versus the overall traffic [32, 52, 21, 8, 14]. In these works, the focus is on providing guarantees on the accuracy of the estimation: the dif-

(19)

ference between the estimation and the actual aggregate is always within a configurable range.

In [8], the authors address continuous queries. In this framework, all sensors hold an estimation of the global aggregate. Changes of this estimation larger than a configurable maximum are propagated in an epidemic fashion. In the evaluation, the authors show that there is a large difference, one order of magnitude, between the actual error and guaranteed the maximum error (the control parameter of the algorithm). This proposal considers only MAX queries and it is unclear how it could be extended to other aggregation functions.

In [52], the authors target periodic n-times queries on MAX, MIN, SUM and AVG aggregates. The proposed algorithm takes into consideration the local variables dy- namics. In each period, nodes report their local variables if and only if the current value differs from the last report in more than a configurable percentage.

As in the previous work, the authors of [52] have found that there is a difference of around one order of magnitude between the mean error and the guaranteed maximum error. Actually, in the evaluation, they consider the relative mean error (over time) rather than the maximum error.

A drawback of the proposal in [52] is that it does not consider the impact of the local changes on the aggregate, but only its relative change. For instance, consider a SUM aggregate with a maximum error of 20%. A local variable (v1) change from 1 to 2 would be reported, while a change from another variable (v2) from 100 to 118 would not be reported. In such a case, for the same use of resources (1 message) the error would be smaller if the reported change was that of v2. Therefore the algorithm in [52] potentially makes a non-optimal use of the resources.

The work in [21] addresses n-time queries, but has some similarities with AGAP:

the aggregate is computed incrementally along a tree, and it uses filters as A-GAP.

Apart from the considered QoE metric, it differs from our work in two aspects. First, it uses synchronized rounds of operation. In contrast, A-GAP is asynchronous. The motivation for this is the achievement of robustness and scalability. Second, the algorithm in [21] requires statistics from all the partial aggregates, which depend on the filters. A-GAP only uses statistics from local variables, which are not affected by filters. This gives A-GAP more flexibility in selecting the duration of the control cycle.

This comes at the cost of assuming independence among the local variables. In some scenarios, independence is a common assumption, e.g.: the number of voice over IP flows entering the domain. It is an open issue how the performance of A-GAP would be affected in case local variables independence would not hold.

The work in [14] also addresses continuous queries. The main difference with the other works is that the aggregates considered are not scalars but frequency distributions. Examples of such distributions in networking are the number of flows per application or the number of flows per destination. This proposal aims at guaranteing that for any element in the domain (e.g., the flow domain), the estimation error is always within a configurable range.

Estimations of frequency distributions permit answering several queries. For the flows per application example, they can provide the number of http flows, the total number of flows or which are the most popular applications. This versatility comes at the cost of a higher overhead than customized solutions for specific queries. For in-

(20)

stance, if the operator is only interested in the number of VoIP and http flows, tailored approaches like [21] would provide such estimates with a lower overhead.

As in [52], the proposal in [14] only considers the relative changes of local variables, not their impact on the aggregate. Therefore, it suffers from the same draw- backs.

A-GAP considers the average error as control parameter, instead of the maximum error. The works in [32, 52] show that, while guaranteeing a maximum error is rela- tively simple, this is a loose upper bound on the accuracy achieved. This is in line with our results, which show that the error at the management station is closer to the average error than to the maximum one. This speaks for using the average absolute error as control parameter. While, in A-GAP, the average error is the control parameter, our protocol can also provide information about the error distribution, including the maximum error, at the management station.

The overhead vs QoE performance achieved by [32], [52], and [21] resembles a negative exponential function for the errors considered, as is the case for A-GAP.

2.3.3 Storage Requirements vs Accuracy in Queries on Data Streams

Another instantiation of the trade-off between QoE and resources is the accuracy vs storage requirements. This is a common trade-off in the context of data streams. A data stream is a sequence of data generated over time by a source. Examples include stock tickers, sequences of clicks in web navigation sessions, periodic readings of sensors, etc.

Queries on data streams commonly refer to the whole stream of data. Providing ex- act answers to them would require storing the whole data stream in the general case.

This would potentially require large amounts of memory, making it impractical in several scenarios. This poses the challenge of answering queries on data streams while minimizing the amount of memory required.

Several researches (see [7] for a survey) have investigated how storage requirements can be traded for the accuracy of the answer. The literature offers a variety of algorithms for controlling this trade-off by creating summaries of the data stream. The accuracy of the information that can be extracted from these summaries is always a function of the summary size. Some examples of such algorithms are:

• Random sampling consists in storing only a subset of the data stream.

Such algorithms implicitly assume that a small subset of the stream con- tains enough information to answer the query. These algorithms can be combined with other summarization techniques.

• Histogram Several algorithms have been proposed for the computation of histograms with memory and accuracy bounds (for a given data set size) . [7] classifies them into three types: (i) piece-wise constant functions, (ii) set of buckets with constant increasing quantiles, and (iii) those aimed at answering iceberg queries [27]. The goal in iceberg queries is to identify data values whose frequency is above a given value. An example of an

(21)

iceberg query is what are the applications whose traffic is more than 5%

of the total traffic.

• Wavelets project a data stream onto a set of vectors. In order to recon- struct the stream perfectly, all the wavelets coefficients are needed. How- ever, the stream can be approximated using the most significant coefficients. It has been shown [7] that wavelets can provide higher accuracy than histograms for the same memory usage.

• Query-specific sketching. Most of the above algorithms are generic in the sense that they can be applied to answering a variety of queries. The literature also provides a number of sketches tailored for specific queries.

Within this group, sketches for queries on frequency moments have attracted a lot of interests. The information provided by the frequency moments includes the number of distinct elements in the data stream, the Gini’s index of homogeneity, and the multiplicity of the most frequent item. The Flajolet and Martin algorithm [28] discussed in section 2.2 falls into this category.

All the algorithms above were originally designed considering n-time queries over a single data stream. The extension of these algorithms to distributed systems could reduce the size of the messages exchanged by including in them a summary of the stream rather than the whole stream. For instance, the Flajolet and Martin algorithm was applied in wireless sensor networks for n-time COUNT [46] and SUM [13] queries. In these works, the stream was composed by the union of the current values of the local variables.

2.3.4 Distributed Streams

Most of the works in data streams consider a single data stream. Multi-streams scenarios have attracted limited attention so far. The most relevant work in this area is that of Gibbons and Tirthapura on distributed streams [30, 31]. In the distributed streams model, there are a number of parties, each of them observing a different data stream. A function of those data streams is provided at a differentiated node.

This model captures the scenarios we consider in the sense that it considers a set of distributed data stream sources in a networked environment and monitoring queries are against the whole set of strems. However, there are a number of differences between our work and [30, 31]. First, these works target n-time queries on past data. For instance, in [30], parties observe the whole data stream before reporting to the referee.

In [31], parties send sketches of its data stream when estimates are requested. Second, the targeted problem space has little relevance in real-time monitoring scenarios. Que- ries on distributed streams do only require significant resources (memory) when the query is computed over a large number of elements of the stream, which is never the case in real-time monitoring, where only the most recent data is of interest. Third, [30] and [31] consider flat topologies. This presents well-known scalability problems.

(22)

A-GAP uses a hierarchical aggregation tree instead. Fourth, the accuracy guarantees are expressed in terms of a maximum error e, which is not exceeded with a probabil- ity p. Both e and p are configurable. The benefits of using the average error as control parameter, as in A-GAP, in front of the maximum error were discussed above. Fi- nally, in [30] the aggregate function considered is the sum of the union of bit streams.

It is unclear the semantics of this aggregate in a distributed network monitoring scenario.

The work in [14] (discussed when considering the trade-off between maximum error and overall traffic ) applies the use of quantile-based summarization techniques to reduce the number of messages in multi-stream scenarios.

(23)

3. Summary of Original Work

Paper A: Design and Implementation of Performance Policies for SMS Systems We present a design for policy-based performance management of SMS Systems.

The design takes as input the operator’s performance goals, which are expressed as policies that can be adjusted at run-time. In our specific design, an SMS administrator can specify the maximum delay for a message and the maximum percentage of messages that can be postponed during periods of congestion. The system attempts to maximize the overall throughput while adhering to the performance policies. It does so by periodically solving a linear optimization problem that takes as input the policies and traffic statistics and computes a new configuration. We show that the computational cost for solving this problem is low, even for large system configurations. We have evaluated the design through extensive simulations in various scenarios. It has proved effective in achieving the administrator’s performance goals and fast in adapting to changing network conditions. A prototype has been developed on a commercial SMS platform, which proves the validity of our design.

This paper has been published in the proceedings of the 16th IFIP/IEEE Distrib- uted Systems: Operations and Management (DSOM 2005), Barcelona, Spain, Octo- ber 24-26, 2005.

Paper B: Distributed Real-time Monitoring with Accuracy Objectives

We introduce A-GAP, a protocol for continuous monitoring of network-wide state variables that achieves a given accuracy with minimal overhead. Network-wide state variables are computed from device counters using aggregation functions, such as SUM, AVERAGE and MAX. In A-GAP, the accuracy is expressed in terms of the average error. A-GAP is decentralized and asynchronous to achieve robustness and scalability. A-GAP runs on an overlay interconnecting management processes on the devices. The protocol follows the push approach to monitoring and uses the concept of incremental aggregation on a self-stabilizing spanning tree. A-GAP achieves monitoring accuracy by dynamically configuring filters in the management processes, which filter out updates to the network-wide state variable. At the same time, it reduces the overhead by attempting to minimize the maximum processing load over all management processes. Filters are computed using a heuristic based on a stochastic model we introduce. We evaluate A-GAP through simulation using an ISP topology (Abovenet) and real traces. The results show that we can effectively control the fundamental trade-off between accuracy and overhead. We specifically show that the protocol overhead can be reduced significantly by al-lowing small error objectives. In

(24)

addition, we show that A-GAP can provide an accurate estimation of the error distribution in real-time.

A short version of this paper has been published in the proceedings of IFIP Net- working 2006, Coimbra, Portugal, May 15-19, 2006.

This paper is available as a KTH Technical Report

Paper C: Scalable Policy Distribution for Ambient Networks

The characteristics of policy-based management make it an interesting candidate for managing Ambient Networks, which are characterized for being highly dynamic and heterogeneous. However, current policy-based approaches are not scalable, which is a must for such dynamic scenarios. A key aspect for developing scalable systems is policy distribution, the mechanism that provides the right policies at the right locations in the network when they are needed. In this paper, we present a scalable framework for policy distribution for Ambient Networks. The framework is based on aggregating the addresses of the policies and applying multipoint communication techniques. The aggregation is based on grouping the managed elements by the role they play in the network and distributing policies that apply to all the elements in a group. We show the validity of the framework by applying it to a study case.

This paper has been published in the 14th IST Mobile and Wireless Communica- tion Summit, Dresden, Germany, June 19-23, 2005

(25)

List of Publications in the Context of this Thesis

1. A. Gonzalez Prieto, R. Stadler "Policy-based Performance Management for SMS Systems," KTH/LCN Technical Report, May 2006

2. A. Gonzalez Prieto, R. Stadler "Distributed Real-Time Monitoring with Ac- curacy Objectives," in the Proceedings of IFIP Networking, Coimbra, Portu- gal, May 15-19, 2006

3. A. Gonzalez Prieto, R. Stadler "Design and Implementation of Performance Policies for SMS Systems," in the Proceedings of the 16th IFIP/IEEE Dis- tributed Systems: Operations and Management (DSOM 2005), Barcelona, Spain, October 24-26, 2005

4. M. Brunner, A. Galis, L. Cheng, J. Andres Colas, B. Ahlgren, A. Gunnar, H.

Abrahamsson, R. Szabo, S. Csaba, J. Nielssen, A. Gonzalez Prieto, R.

Stadler, G. Molnar, "Towards Ambient Networks Management," Second In- ternational Workshop on Mobility Aware Technologies and Applications (MATA 2005), Montreal, Canada, October 17-19, 2005

5. A. Gonzalez Prieto, R. Stadler, "Scalable Policy Distribution for Ambient Networks," 14th IST Mobile and Wireless Communication Summit, Dres- den, Germany, June 19-23, 2005

6. A. Gonzalez Prieto, R. Stadler "Policy-based Management for SMS Sys- tems," RVK 2005, Linkoping, Sweden, June 14-16, 2005

7. A. Gonzalez Prieto, R. Stadler "Evaluating a Congestion Management Archi- tecture for SMS Gateways," 9th IFIP/IEEE International Symposium on In- tegrated Network Management (IM 2005), Nice, France, May 15-19, 2005 (poster)

8. M. Brunner, A. Galis, L. Cheng, J. Andres Colas, B. Ahlgren, A. Gunnar, H.

Stadler, G. Molnar, "Ambient Networks Management Challenges and Ap- proaches," First International Workshop on Mobility Aware Technologies and Applications (MATA 2004), Florianpolis, Brazil, October 20-22, 2004 9. A. Gonzalez Prieto, R. Cosenza and R. Stadler "Policy-based Congestion

Management for an SMS Gateway," in the Proceedings of the IEEE 5th In- ternational Workshop on Policies for Distributed Systems and Networks (POLICY 2004), Yorktown Heights, New York, June 7-9, 2004

(26)

(27)

4. Key Results and Research

In this section, we present the results of the three lines of work presented in this thesis. Our intention is to focus our future research on extending the work on decentralized real-time monitoring. For this purpose, we conclude the section with a set of research questions on this topic.

4.1 Policy-based Performance Management for SMS Systems

• We have presented a policy-based design for performance management of SMS systems

• Our design facilitates the management of messaging gateways by raising the level of abstraction of the interaction with the design. This permits administrators with a basic understanding of performance metrics to control a gateway. They are not required to have a detailed knowledge of the device internals.

• The design has been evaluated through extensive simulations in several case studies. The experiments show that:

o the design performs remarkably close the administrator’s performance goals

o the computational cost of the policy evaluation is low, even for large system configurations.

o the system is not very sensitive to the traffic characteristics of the offered load.

• A prototype was developed on the EMG gateway. This shows the feasibil- ity of implementing our design on a commercial platform.

4.2 Distributed Real-time Monitoring with Accuracy Objectives

• We have presented a protocol for continuous monitoring with accuracy objectives: A-GAP.

• Key aspects of the protocol are:

o it uses the concept of incremental aggregation on a spanning tree o it follows a push approach

(28)

o it is decentralized and asynchronous (for robustness and scalability purposes)

o it adapts to changes in the statistics of the local variables o it adapts to failures and topology changes

• A-GAP has been evaluated through simulation. The experiments show that it:

o effectively controls the trade-off between accuracy and overhead, o reduces the overhead significantly when allowed some estima-

tion error,

o the estimation error by A-GAP exceeds the error objective by a (small) margin. This is due to networking and processing delays.

Estimating this margin in real-time, based on the evolution of local variables and processing/communication delays, would permit tuning the protocol.

• Accuracy objectives are expressed in terms of the average absolute error.

We argue this parameter is more significant for practical applications than the objective of a maximum error, suggested in the recent literature.

• A-GAP enables performance prediction at run-time. Based on our stochastic model, a manager can be provided (for a given accuracy objective) with:

o an estimation of the expected load on all nodes. This is potentially significant in real scenarios. For instance, a manager could avoid overloading the management system by loosening the error objective,

o the distribution of the estimation error at the root node.

4.3 Scalable Policy Distribution

• We have shown that current approaches to policy distribution cannot cope with dynamic scenarios.

• We have presented a scalable framework for policy distribution

• Key aspects of the framework are:

o the aggregation of the addresses of the policies based on their role

o the application of multipoint communication techniques

(29)

• Our proposal scales with the unicast routing algorithm deployed in the network.

• The validity of the approach has been shown by applying it to a realistic scenario.

• Our proposal is consistent with the IETF framework.

4.4 Research Questions

We present a set of questions and open issues regarding future work on decentralized real-time monitoring with data aggregation and accuracy objectives.

• How can A-GAP be extended to support different aggregation functions?.

Some examples of interest are iceberg queries and statistical aggregates of local variables.

• How can A-GAP be extended to operate efficiently in wireless and mobile environments?

• A-GAP enables performance prediction at run-time, based on our stochastic model. How can A-GAP be extended such that a network administrator can answer what if... questions such as: if I set the allowed average error to X, what would be the overhead? or what accuracy can I get while keep- ing the overhead below Y ?

• A-GAP attempts to solve a network-wide optimization problem in a distributed fashion. Finding a centralized algorithm for solving the network- wide problem would provide a benchmark for A-GAP and help in improving the design of A-GAP.

• In A-GAP, each node solves a local optimization problem. The solution is determined based on a grid search. Are there alternative and faster algorithms for solving the local problem?

• The stochastic model used in A-GAP assumes independence among local variables. In many monitoring scenarios this assumption is not realistic.

What is the impact of correlated variables in the performance of A-GAP?

• Given an overlay, A-GAP creates and maintains a BFS aggregation tree.

Clearly, the performance of A-GAP is affected by the topology of the overlay. What overlay topologies improve the performance of A-GAP?

• A-GAP, currently, re-configures the node filters periodically. Alterna- tively, this re-configuration could be triggered by external events. This

(30)

approach could significantly reduce the overhead or increase the adaptiv- ity.

• A-GAP, currently, is based on the use of filters that drop updates with small variations of partial aggregates. An alternative to using filters is to limit the rate of updates a node can send. It is an open issue how to develop a rate-based protocol with accuracy objectives and how it would compare to A-GAP.

• At this point, A-GAP has been evaluated based on simulations. What can we learn from a testbed implementation?

• Security. In A-GAP, nodes cooperate to achieve a goal. Faulty and mali- cious nodes may jeopardize the achievement of the goal. A-GAP would require mechanisms to identify and isolate such nodes.

(31)

Bibliography

[1] Coin: Computational infrastructure for operations research.

http://www.coinor.org/index.html, May 2005.

[2] Cygwin. http://cygwin.com, May 2005.

[3] GSM association, www.gsmworld.com, May 2005.

[4] Mysql. http://www.mysql.com, May 2005.

[5] Nordic messaging. www.nordicmessaging.se, May 2005.

[6] ANSI/IEEE Std 802.1D, 1998.

[7] B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom, “Models and issues in data stream systems”, In 21st ACM Symposium on Principles of Da- tabase Systems, Madison, USA, June 2002.

[8] A. Boulis, S. Ganeriwal, and M. B. Srivastava, “Aggregation in sensor networks: an energy - accuracy tradeoff”, Elsevier Ad-hoc Networks Journal (special issue on sensor network protocols and applications), pages 317–331, 2003.

[9] D. Breitgand, D. Dolev, and D. Raz, “Accounting mechanism for member- ship size-dependent pricing of multicast traffic”, In Internet Charging and QoS Technology (ICQT), Munich, Germany, September 2003.

[10] M. Brunner, A. Galis, L. Cheng, J. Andres Colas, B. Ahlgren, A. Gunnar, H.

Stadler, and G. Molnar, “Ambient networks management challenges and approaches”, In First International Workshop on Mobility Aware Technologies and Applications (MATA 2004), Florianpolis, Brazil, August 2004.

[11] M. Brunner, A. Galis, L. Cheng, J. Andres Colas, B. Ahlgren, A. Gunnar, H.

Stadler, and G. Molnar, “Towards ambient networks management”, In Sec- ond International Workshop on Mobility Aware Technologies and Applica- tions (MATA 2005), Montreal, Canada, October 2005.

[12] D. Cavendish, M. Gerla, and S. Mascolo, “A control theoretic approach to congestion control in packet networks”, IEEE/ACM Transactions on Net- working, 12(5), 2004.

(32)

[13] J. Considine, F. Li, G. Kollios, and J. Byers, “Approximate aggregation techniques for sensor databases”, In 20th International Conference on Data Engi- neering, Boston, USA, March 2004.

[14] G. Cormode, M. Garofalakis, S. Muthukrishnan, and R. Rastogi, “Holistic aggregates in a networked world: distributed tracking of approximate quantiles”, In ACM SIGMOD International Conference on Management of Data, Baltimore, USA, June 2005.

[15] A. Corrente, M. De Bernardi, and R. Rinaldi, “Policy provisioning performance evaluation using COPS-PR in a policy based network”, In 8th IFIP/IEEE International Symposium on Integrated Network Management (IM 2003), Colorado Springs, USA, March 2003.

[16] F. R. Cosenza, “Enterprise messaging gateway management”, Master’s thesis, KTH Royal Institute of Technology, April 2005.

[17] S. Coulombe and G. Grassel, “Multimedia adaptation for the multimedia messaging service”, IEEE Communications, 42(7), July 2004.

[18] Y. K. Dalal and R. M. Metcalfe, “Reverse path forwarding of broadcast pack- ets”, Communications of the ACM, 21(12), December 1978.

[19] M. Dam and R. Stadler, “A generic protocol for network state aggregation”, In Radiovetenskap och Kommunication (RVK), Linkoping, Sweden, June 2005.

[20] G.B. Dantzig, “Maximization of linear function of variables subject to linear inequalities”, In Activity Analysis of Production and Allocation, pages 339–

347. 1951.

[21] A. Deligiannakis, Y. Kotidis, and N. Roussopoulos, “Hierarchical in-network data aggregation with quality guarantees”, In 9th International Conference on Extending Database Technology (EDBT), Crete, Greece, March 2004.

[22] C. Diot, W. Dabbous, and J. Crowcroft, “Multipoint communications: A survey of protocols, functions, and mechanisms”, IEEE Journal on Selected Ar- eas in Communications (J-SAC), pages 277–90, 1997.

[23] S. Dolev, A. Israeli, and S. Moran, “Self-stabilization of dynamic systems assuming only read/write atomicity”, Distributed Computing, 7:3–16, 1993.

[24] A. Westerinen et al., “RFC 3198: Terminology for policy-based management”, November 2001.

(33)

[25] N. Niebert et al., “Ambient networks - research for communication networks beyond 3G”, In 13th IST Mobile and Wireless Communications- Summit 2004, Lyon, France, June 2004.

[26] S. Floyd et al., “A reliable multicast framework for light-weight sessions and application level framing”, IEEE ACM Transactions on Networking, 5(6), 1997.

[27] M. Fang, N. Shivakumar, H. Garcia-Molina, R. Motwani, and J. D. Ullman,

“Computing iceberg queries efficiently”, In 24th Int. Conf. Very Large Data Bases (VLDB), New York City, USA, August 1998.

[28] P. Flajolet and G. N. Martin, “Probabilistic counting algorithms for data base applications”, Journal of Computer and System Sciences, 31(2):182–209, October 1985.

[29] S. Floyd and V. Jacobson, “Random early detection gateways for congestion avoidance”, IEEE/ACM Transactions on Networking, 1993.

[30] P. B. Gibbons and S. Tirthapura, “Estimating simple functions on the union of data streams”, In ACM Symposium on Parallel Algorithms and Architec- tures, Crete Island, Greece, June 2001.

[31] P. B. Gibbons and S. Tirthapura, “Distributed streams algorithms for sliding windows”, In ACM Symposium on Parallel Algorithms and Architectures, Winnipeg, Canada, August 2002.

[32] C. Intanagonwiwat, D. Estrin, R. Govindan, and J. Heidemann, “Impact of network density on data aggregation in wireless sensor networks”, In 22^nd In- ternational Conference on Distributed Computing Systems, Vienna, Austria, July 2002.

[33] C. Intanagonwiwat, R. Govindan, and D. Estrin, “Directed diffusion: A scalable and robust communication paradigm for sensor networks”, In Sixth An- nual International Conference on Mobile Computing and Networking (Mobi- com), Boston, USA, August 2000.

[34] V. Klee and G.J. Minty, “How good is the simplex algorithm?”, In Inequali- ties III, pages 159–175. Academic Press, 1972.

[35] B. Krishnamachari, D. Estrin, and S. Wicker, “The impact of data aggregation in wireless sensor networks”, In International Workshop of Distributed Event-Based Systems, Vienna, Austria, July 2002.

[36] B. Krishnamachari, D. Estrin, and S. Wicker, “Modelling data-centric routing in wireless sensor networks”, Technical Report CENG 02-14, USC Com- puter Engineering, 2002.

(34)

[37] H.T. Kung and R. Morris, “Credit-based flow control for ATM networks”, IEEE Network Magazine, pages 40–48, March-April 1995.

[38] K. S. Lim and R. Stadler, “A navigation pattern for scalable internet management”, In IEEE/IFIP IM 2001, Seattle, USA, May 2001.

[39] K. S. Lim and R. Stadler, “Weaver - realizing a scalable management paradigm on commodity routers”, In IFIP/IEEE IM 2003, Colorado Springs, USA, March, 2003.

[40] K. S. Lim and R. Stadler, “Simpson - a simple pattern simulator for networks”, http://www.comet.columbia.edu/adm/software.htm, 2006.

[41] A. Liotta, G. Pavlou, and G. Knight, “Exploiting agent mobility for large scale network monitoring”, IEEE Network, special issue on Applicability of Mobile Agents to Telecommunications, 16(3), May/June 2002.

[42] S.R. Madden, M.J. Franklin, J.M. Hellerstein, and W. Hong, “TAG: a tiny aggregation service for ad-hoc sensor networks”, In Fifth Symposium on Op- erating Systems Design and Implementation, Boston, USA, December 2002.

[43] A. Mankin and K. Ramakrishnan, “RFC 1254: Gateway congestion control survey”, August 1991.

[44] M.J. Maullo and S.B. Calo, “Policy management: an architecture and approach”, In First IEEE International Workshop on Systems Management, Los Angeles, USA, April 1993.

[45] J.D. Moffett and M.S. Sloman, “Policy hierarchies for distributed systems management”, IEEE Journal on Selected Areas in Communications (J-SAC), 11(3), 1993.

[46] S. Nath, P. B. Gibbons, S. Seshan, and Z. Anderson, “Synopsis diffusion for robust aggregation in sensor networks”, In Second ACM Conference on Em- bedded Networked Sensor Systems, Baltimore, USA, November 2004.

[47] C. Olston, J. Jiang, and J. Widom, “Adaptive filters for continuous queries over distributed data streams”, In ACM SIGMOD 2003, San Diego, USA, June 2003.

[48] C. Olston, B. T. Loo, and J. Widom, “Adaptive precision setting for cached approximate values”, In ACM SIGMOD 2001, Santa Barbara, USA, May 2001.

(35)

[49] C. Olston and J. Widom, “Efficient monitoring and querying of distributed, dynamic data via approximate replication”, IEEE Data Engineering Bulletin, March 2005.

[50] R. Perlman. Interconnections. Addison Wesley Longman, 2000.

[51] A. Polirakis and R.Boutaba, “The meta-policy information base”, IEEE Net- work, special issue on Policy-Based Networks, 16(2):40–48, 2002.

[52] M. A. Sharaf, J. Beaver, A. Labrinidis, and P. K. Chrysanthis, “Balancing energy efficiency and quality of aggregate data in sensor networks”, ACM In- ternational Journal on Very Large Data Bases, 13(4):384–403, December 2004.

[53] M. Sloman, “Policy driven management for distributed system”, Journal of Networks and Systems Management, 2(4), 1994.

[54] I. Solis and K. Obraczka, “The impact of timing in data aggregation for sensor networks”, In International Conference on Communications (ICC), Paris, France, June 2004.

[55] D.A. Spielman and S. Teng, “Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time”, In ACM Symposium on The- ory of Computing, Crete, Greece, July 2001.

[56] N. Spring, R. Mahajan, and D. Wetherall, “Measuring ISP topologies with Rocketfuel”, In ACM/SIGCOMM, Pittsburgh, USA, August 2002.

[57] D. Verma, “Simplifying network administration using policy-based management”, IEEE Network, special issue on Policy-Based Networks, 16(2):20–26, 2002.

[58] D. Waitzman, C. Partridge, and S. Deering, “RFC 1075: Distance vector multicast routing protocol”, November 1998.

[59] J. Zhao, R. Govindan, and D. Estrin, “Computing aggregates for monitoring wireless sensor networks”, In First IEEE International Workshop on Sensor Network Protocols and Applications, Anchorage, USA, May 2003.

Adaptive management for networked systems