3.3 Case: A DHT testbed
3.3.2 Monitor configuration
When Vendetta is initiated, it reads its configuration file and present the GUI shown in figure 3.1. The part of the configuration file that decides the look of the monitor is presented in figure 3.4. To be able to set the background color has been very convenient when we have changed between using monitors and projectors to present the visualization.
The <CANVAS> tag is used to specify what canvases to use in the GUI - the
DHT ring canvas and the globe canvas that have already been introduced.
The canvas area is shared equally between the canvases so removing the DHT canvas from the configuration file would leave all of the top of the screen to the globe.
The bottom left part of the GUI includes a node table, with the columns defined in the <TABLECOLUMN> tag in the configuration file, as well as a status message window.
The <NODECMD>, <OVERLAYCMD> and <LOGFILTER> tags are used to define the management commands available in the GUI. When the configuration is parsed by Vendetta, it creates the command panels shown in figures 3.8,3.6, and 3.10.
Node client commands
The configuration of the node commands needed in our case study is pre-sented in figure 3.5. We have defined node commands to be commands that would be needed for most testbeds, but which are not directly affecting the tested application. Node commands can for example start or stop the remote node client, update the software running on the remote nodes, or flush the remote message queue. From the configuration file we can see that updating the node client software is done using a local script. The script simply con-tains a rsync command. In the configuration of commands the user can use
<NODE HOSTNAME> which will be replaced with the hostname of the chosen node. If multiple nodes are chosen it will result in a number of parallel calls.
A limited amount of parallel calls are allowed, so if too many nodes are called at once the calls are buffered to not overload the monitor. To start the node client, a remote shell command is done using SSH. When the node client is started, TCP can be used to send commands. For example the command to clear the node clients message buffer is using TCP.
Application specific commands
In the next tab, there are the commands that directly affects the tested ap-plication. In our case it is to start and stop the Bamboo overlay on a node, to request the routing table, or to request the leaf set. The configuration needed to create the buttons are presented in figure 3.7. From the configuration file we can see that all overlay commands are sent using TCP to the node client.
At the node client messages are parsed by the corresponding Java classes.
We use commands over TCP rather than SSH because the SSH daemon on the PlanetLab nodes can have large response times. Response times of up to 20 seconds is not uncommon and when we want to control node behavior in
label=Start VClient type=EXEC
msg=ssh uu_bamboo@<NODE_HOSTNAME> daemonize.pl pandora.kicks-ass.org:4444
label=Kill Monitor Node type=TCP
label=pkill java type=EXEC
msg=ssh uu_bamboo@<NODE_HOSTNAME> pkill java
label=Update VClient type=EXEC
msg=TIMEOUT=60000 ./scripts/updateclient <NODE_HOSTNAME>
label=Update Bamboo type=EXEC
msg=TIMEOUT=600000 ./scripts/updatebamboo <NODE_HOSTNAME>
label=Run Node Script type=TCP
label=Clear Node Queue type=TCP
Figure 3.5: Configuration of node client commands
Figure 3.6: Node commands panel
label=Start Bamboo type=TCP
msg=CTRL_NET_UP_REQ args=run-java args=bamboo.lss.DustDevil args=node.cfg
label=Kill Bamboo type=TCP
label=Request RT type=TCP msg=CTRL_RT_REQ
label=Request Leafset type=TCP
Figure 3.7: Application specific commands
Figure 3.8: Overlay commands panel
Put Started:LE_PUT_STARTED Put Forwarded:LE_PUT_FORWARD Put Delivered:LE_PUT_DELIVERED Got Key from Root:LE_GOT_KEY_FROM_ROOT num Stored Keys:LE_STORED_KEYS
Figure 3.9: The set of possible log
events Figure 3.10: Log filter panel
detail the extra delay of SSH is problematic.
The configuration file also defines the possible log events, shown in figure 3.9. The log events can be either sent to the monitor using UDP or TCP, or only logged locally by the node client. In the corresponding panel, shown in figure 3.10, there are drop down boxes containing the three different log event actions. When initiated they do not show the current state of the log event, because the default behavior is defined in the node clients configuration file.
If the user change the behavior with the drop down menus they will then show the current state.