Astronomy Software Integration with OpenSpace

Full text

(1)LiU-ITN-TEK-A--20/062-SE. Astronomy Software Integration with OpenSpace Aniisa Bihi Johanna Granström 2020-12-10. Department of Science and Technology Linköping University SE-601 74 Norrköping , Sw eden. Institutionen för teknik och naturvetenskap Linköpings universitet 601 74 Norrköping.

(2) LiU-ITN-TEK-A--20/062-SE. Astronomy Software Integration with OpenSpace The thesis work carried out in Medieteknik at Tekniska högskolan at Linköpings universitet. Aniisa Bihi Johanna Granström Norrköping 2020-12-10. Department of Science and Technology Linköping University SE-601 74 Norrköping , Sw eden. Institutionen för teknik och naturvetenskap Linköpings universitet 601 74 Norrköping.

(3) Upphovsrätt Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/ Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/. © Aniisa Bihi, Johanna Granström.

(4) Linköping University | Department of Science and Technology Master thesis, 30 ECTS | Media Technology and Engineering Fall 2020 | LiU-ITN-TEK-A–20/062-SE. Astronomy Software Integration with OpenSpace Integration av astronomiska mjukvaror med OpenSpace. Aniisa Bihi Johanna Granström Supervisor: Emma Broman Examiner: Alexander Bock. Linköping University SE-601 74 Norrköping +46 13 28 10 00, www.liu.se.

(5) Abstract This thesis aimed to create a messaging protocol for OpenSpace to interoperate with other astronomy software. An existing proof of concept between OpenSpace and the software Glue was implemented further. The goal was to create a messaging standard that was not language dependent and could be implemented by any astronomy software. To establish an asynchronous communication between OpenSpace and connected software, the Transmission Control Protocol (TCP), threading, and Peer-To-Peer (P2P) were the techniques mainly used. TCP was used to achieve reliable communication between software connected to the network. The enabling of two-way communication was solved by threading. P2P was used as a network communication architecture to share resources between the connected software. By using Unicode characters expressed through UTF-8, the Unicode Standard was used to encode messages sent. The messages are structured by combinations of different sizes of bytes and are sent and received as binary strings. All messages contain a header and the data being sent. Different message types were created to specify which type of data is sent. The protocol works primarily between OpenSpace and Glue but is not limited to these software. The implementation serves as the basis of the messaging protocol for OpenSpace, where Glue represents future software integrations. The master thesis project resulted in a messaging protocol called Software Integration Messaging Protocol (SIMP) for OpenSpace. It consists of a messaging standard based on declared message types sent as binary strings through TCP-sockets. It was concluded that the messaging protocol for OpenSpace could be used by other software than Glue. The protocol is not language or software dependent and can be expanded to include more message types and functionalities..

(6) Acknowledgements This project has had its highs and lows but has continuously pushed us to greater lengths. We have had such a fun and great time working with OpenSpace and would like to thank the fantastic people that helped us along the way. Thank you to our supervisor Emma who has always been there to help when we needed it. From debugging code to teaching us the structure of OpenSpace, you have taught us sufficient ways to learn new things. Thank you for all your support Emma and for always being there for us. Thank you to Alex that has been more than an examiner to us. You have taken the time of your busy schedule to make sure we have everything we need and explained the ins and outs of OpenSpace. You truly are the heart of OpenSpace. To Chuck and Gene, even though we could not visit the SCI Institute, you both have been a tremendous help during our thesis. Hopefully, one day we can all meet in person and visit the SCI Institute. Gene, despite the time difference, you have helped us navigate complex code and concepts and have always been a Slack message away. Chuck, thank you for all your help and support and for offering us your home to stay in when we were planning to come to Utah. We would have loved to stay with you and your wife, say hi to your cat from us. Jackie, thank you for being the perfect user subject. Thank you for taking the time to help us create a tool that the astronomy community is excited about. Your excitement and joy have fueled us in times of discouragement. To our friends and family, thank you for your endless support over these months. There is nothing better than having wonderful people in your life who supports you. To Johanna, thank you for being the absolute best thesis partner and friend I could imagine. Love, Aniisa. To Aniisa, thank you for being such an understandable and caring partner and friend one could wish for. Love, Johanna.. We are graduating!!!! Aniisa & Johanna November 2020.

(7) Contents. 1 Introduction. 1. 1.1. OpenSpace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 1. 1.2. Glue: Multi-Dimensional Linked Data Exploration . . . . . . . . . . .. 2. 1.3. The Dagstuhl Seminar 19262 . . . . . . . . . . . . . . . . . . . . . . .. 2. 1.4. Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 3. 1.4.1. Research Questions . . . . . . . . . . . . . . . . . . . . . . . .. 3. 1.4.2. User Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 3. Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 3. 1.5. 2 Related Work. 4. 2.1. Simple Application Messaging Protocol . . . . . . . . . . . . . . . . .. 4. 2.2. Glue-OpenSpace Plugin . . . . . . . . . . . . . . . . . . . . . . . . .. 5. 3 Theory. 6. 3.1. Transmission Control Protocol . . . . . . . . . . . . . . . . . . . . . .. 6. 3.2. Peer-To-Peer Networking . . . . . . . . . . . . . . . . . . . . . . . . .. 7. 3.3. Threading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 7. 3.4. Unicode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 8. 3.4.1. UTF-8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 8. OpenSpace Architecture . . . . . . . . . . . . . . . . . . . . . . . . .. 9. 3.5. 4 Implementation 4.1. 10. Implementation Goals . . . . . . . . . . . . . . . . . . . . . . . . . . 10.

(8) 6. Contents. 4.2. Messaging Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.2.1. Header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. 4.2.2. Connect (CONN) . . . . . . . . . . . . . . . . . . . . . . . . . 13. 4.2.3. Disconnect (DISC) . . . . . . . . . . . . . . . . . . . . . . . . 13. 4.2.4. Point Data (PDAT) . . . . . . . . . . . . . . . . . . . . . . . . 13. 4.2.5. Add Scene Graph Node (ASGN) . . . . . . . . . . . . . . . . . 14. 4.2.6. Remove Scene Graph Node (RSGN). 4.2.7. Update Color (UPCO) . . . . . . . . . . . . . . . . . . . . . . 16. 4.2.8. Update Opacity (UPOP) . . . . . . . . . . . . . . . . . . . . . 16. 4.2.9. Update Size (UPSI) . . . . . . . . . . . . . . . . . . . . . . . . 17. . . . . . . . . . . . . . . 15. 4.2.10 Toggle Visibility (TOVI) . . . . . . . . . . . . . . . . . . . . . 17 4.3. Receiving Messages in OpenSpace . . . . . . . . . . . . . . . . . . . . 18 4.3.1. Rendering Received Point Data in OpenSpace . . . . . . . . . 18. 4.4. Sending Messages from OpenSpace . . . . . . . . . . . . . . . . . . . 19. 4.5. Testing the Protocol with Glue . . . . . . . . . . . . . . . . . . . . . 19. 4.6. 4.5.1. Sending Messages . . . . . . . . . . . . . . . . . . . . . . . . . 19. 4.5.2. Sending Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 20. 4.5.3. Receiving Messages . . . . . . . . . . . . . . . . . . . . . . . . 21. User Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21. 5 Result 5.1. 22. User Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29. 6 Discussion. 30. 6.1. Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30. 6.2. Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32. 6.3. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33.

(9) 7 Conclusion. 35. A SAMP. 37. A.1 Simple Application Messaging Protocol . . . . . . . . . . . . . . . . . 37 A.1.1 MTypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 A.1.2 C Library: Libsamp. . . . . . . . . . . . . . . . . . . . . . . . 38. A.2 Disadvantages Of SAMP . . . . . . . . . . . . . . . . . . . . . . . . . 39 A.2.1 Developers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 A.2.2 C-Library: Libsamp. . . . . . . . . . . . . . . . . . . . . . . . 39. A.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 B Software Integration Messaging Protocol for OpenSpace. 41. C User Test Instructions. 54. D Questionnaire. 58. References. 66.

(10) Chapter 1 | Introduction. This master thesis was a collaboration between Linköping University (LiU) and the Scientific Computing and Imaging Institute of Utah (SCI). The goal was to improve the integration of existing astronomical analysis software to OpenSpace [1]. One of the main aspects of OpenSpace is the ability to provide astronomy and astrophysics researchers with a tool to bring their data and findings to the public. There have been a few areas in which this has been accomplished in OpenSpace. Therefore, this thesis aims at solving the problem on a larger scale by improving the integration with other astronomical software. The software integration focused on the software called Glue [2]. Astronomers widely use Glue to analyze and visualize data sets. However, they need tools to show the results in an immersive 3D environment, which OpenSpace aims to achieve.. 1.1. OpenSpace. OpenSpace is an open-source software project funded in part by NASA that utilizes the latest data visualization research techniques. OpenSpace is used to visualize the entire known universe and create an interactive environment. Data is gathered from previous and ongoing space missions, observations and simulations. OpenSpace aims to promote a greater understanding of Earth and space science research by intuitively illustrating the known universe. Through visualizations, the public can learn about NASA programs and academic research, which is the primary purpose of OpenSpace. The project is a collaboration between Linköping University, NASA Goddard Community Coordinated Modeling Center, American Museum of Natural History, the Visualization and Data Analysis Lab at New York University Tandon School of Engineering and the Scientific Computing and Imaging Institute at the University of Utah [3]. The SCI is a world-leading visualization research laboratory and a long-standing partner to the OpenSpace project. The SCI research involves the development of innovative and robust software packages. Software at the SCI is developed in close collaboration with users to satisfy needs within research communities [4].. 1.

(11) 2. 1.2. Chapter 1. Introduction. Glue: Multi-Dimensional Linked Data Exploration. Glue is a visualization tool used by astronomers and researchers to visualize astronomy data, among others. Glue is an open-source Python library and application used to explore relationships within and between related data sets. Using Glue, users can visualize their data, both in 2D and 3D, with scatter plots, histograms, and images. Users are allowed to integrate their Python code for data input, cleaning, and analysis. Glue focuses on brushing and linking patterns, where selections in one graph show in all other connected graphs [5]. The concepts used to go from a selection on the screen to a defined subset are regions of interest, subsets states and data subsets. A Region of Interest (ROI) is an abstract representation of a geometrical region or selection identified for a particular purpose. A data subset results from applying a ROI to a specific data set, and a subset state describes a data subset and its properties. When a user selects data in a data viewer in the Glue application, the selection is first translated into a ROI. The ROI then converts to a subset state and is applied to the data collection to produce subsets in each data set [5]. By integrating this kind of software to OpenSpace, astronomers, and researchers can bring their expert knowledge to the public. An optional approach would be to implement all the available visualization techniques into one tool. The problem is that there are many expert tools out there that can be used to a very high degree. Combining all to one tool would mean that researchers have to learn a whole new software, and it would also signify much work to implement and maintain. Therefore it is more profitable to be able to operate between different applications.. 1.3. The Dagstuhl Seminar 19262. The concept of software integration with OpenSpace came from the Dagstuhl seminar in June of 2019 called Astrographics: Interactive Data-Driven Journeys through Space [6]. The seminar brought together researchers from computer science, content producers, learning and communication experts, and domain experts from astronomy and astrophysics. The aim was to define the emerging field of interactive visualization of space exploration and astronomy, referred to as Astrographics. The Dagstuhl seminar discussed how to design available visualization software to serve both exploratory and explanatory visualizations. Exploratory visualizations visualize scientific discoveries, and explanatory visualizations communicate scientific discoveries to a general audience. The seminar consisted of discussions in break-out groups, which consequently led to integrating Glue with OpenSpace. Using OpenSpace with visualization software will let researchers perform advanced data analysis, link data sets together, and explore subsets of data while visualizing the data in a realistic rendering of the universe. The Dagstuhl seminar resulted in a proof of concept for integrating Glue to OpenSpace, which later gave the idea for this thesis [6]..

(12) 1.5. Delimitations. 1.4. 3. Aim. This master thesis project aims to create a software integration messaging protocol designed to transfer data from astronomical analysis software to OpenSpace. The OpenSpace tool should be developed for astronomers, to enable the possibility to show results in an immersive 3D environment.. 1.4.1. Research Questions. Answers to the following research questions will be sought and given. • How could a software integration tool for OpenSpace achieve interoperability with astronomy software such as Glue? • How could the software integration be implemented to manage the exchange of large data sets between OpenSpace and astronomy software? • How should the messaging protocol of the software integration be implemented, in an intuitive way, to make astronomy software developers understand what interoperability operations are possible to perform?. 1.4.2. User Needs. The needs of future users are taken into consideration to help achieve the aim of this thesis. The target group for this project is astronomical researchers. The following three functionalities are the main functionalities requested by users. • The ability to send/receive data between OpenSpace and astronomy software • The ability to select and highlight data in one application, e.g. Glue, and having the same data highlighted in OpenSpace • The ability to look at astronomical data in one application and then use OpenSpace to see the location of that data in space. 1.5. Delimitations. Although the aim was to make OpenSpace interoperable with other software tools, the focus was on making the communication work primarily for OpenSpace. This focus suggests that other software will have to implement or update their messaging standards to follow the OpenSpace messaging protocol..

(13) Chapter 2 | Related Work. There are several existing ways to send messages for communication between software. The most common is the communication protocol Hypertext Transfer Protocol (HTTP), which is used for queries on the World Wide Web. A goal with messaging protocols is to achieve seamless communication and with no limit to the data size. Astronomy software usually aim to follow Virtual Observatory (VO) standards [7]. The VO standard originates from the vision that astronomical data sets, software, and other resources should work as a seamless whole. Instead of focusing on creating software or data sets that can reach all user needs, the aim is to allow different software to interoperate using the same data sets [8]. With this aim in mind, members of the International Virtual Observatory Alliance (IVOA) created a messaging protocol called the Simple Application Messaging Protocol (SAMP) [9].. 2.1. Simple Application Messaging Protocol. SAMP is an existing messaging protocol used to communicate between astronomical visualization tools such as Glue, World Wide Telescope [10], TOPCAT [11], and Aladin [12]. SAMP is an abstract framework with a Remote Procedure Call (RPC) based structure. RPC based communication is a protocol used to request information from another application or computer via different calls. It is built to be language-independent and is therefore making few assumptions about the programming language used. SAMP includes event-based communication, based on a central service providing a multi-directional publish and subscribe message broker. SAMP follows a hub-based architecture that creates a network node with several Input/Output (I/O) ports for clients to connect and communicate through. To define which messages to receive, the client then subscribes to a list of different message types and sends or receives messages to or from other clients via the hub. Each message type is characterized by a string that defines the semantics of a message and its arguments.. 4.

(14) 2.2. Glue-OpenSpace Plugin. 5. Sending messages can either be done via asynchronous or synchronous calls and responses. Synchronous messaging implies that the sender waits for a reply before continuing with tasks. Asynchronous messaging continues with tasks regardless if a reply has been received or not. In addition to asynchronous and synchronous messaging, a third messaging pattern is called Notification. This pattern implies a strictly one-way communication and that the receiver provides no feedback to the sender. The hub implementation SAMP uses offers all three messaging pattern alternatives.. 2.2. Glue-OpenSpace Plugin. The development of an experimental plugin for OpenSpace and Glue was initiated at the Astrographic: Interactive Data-Driven Journeys through Space workshop at the Dagstuhl seminar of 2019. The plugin was a proof of concept of how to use OpenSpace with the Glue data visualization software. The plugin allows sending data from Glue to OpenSpace. The user can make changes to the data in Glue and send those changes via messages to OpenSpace. The corresponding changes then occur in OpenSpace. For example, if the color property of the data set in Glue is updated, the same color update will occur in OpenSpace. By using WebSockets, a localhost is created as a connection point with a specified port for OpenSpace and Glue to communicate through. WebSockets are point-topoint systems which refer to communication endpoints and offers both synchronous and asynchronous pattern design. This plugin uses the asynchronous pattern, implying that Glue does not wait for OpenSpace to send a reply before sending a new message. Lua scripting is used in OpenSpace to call and test the functionality of functions. These calls are made via Lua script commands. For example, a Lua command can from a specified table load the scene graph node described in the table and add it to the scene graph in OpenSpace. To send messages from Glue to OpenSpace, Glue sends different Lua script commands as messages. OpenSpace receives the message containing the Lua script and makes a call to that command. When sending data from Glue to OpenSpace, the plugin creates a temporary local file on the computer and tells OpenSpace in a message where to find the file. OpenSpace can, in turn, find the temporary file, parse the data and visualize it..

(15) Chapter 3 | Theory. Existing methods and techniques can be used to ease the implementation of a messaging protocol. To better understand how to use them, it is necessary to understand what utility they would bring to this master thesis project. The techniques used will be described in sufficient detail to understand their relevance for the project.. 3.1. Transmission Control Protocol. The Transmission Control Protocol (TCP) provides reliable communication between applications connected to a network. It is an end-to-end protocol that is designed to fit into a hierarchy built up by different layers. The TCP fits into the layer above an Internet Protocol (IP) layer (see Table 3.1). The IP layer is the layer that enables the TCP layer to send and receive messages. It also carries security classification so the messages can be communicated end-to-end across multiple networks. The TCP and internet protocol work together to provide calls made by the user on the TCP to open or close a connection, send or receive data, or to give an status of the connection made. The application level contains higher level applications such as file transfer protocols, which is a set of rules that a computer follows for transferring files from one system to another via the internet [13]. Application-level TCP IP Communication network Table (3.1): Protocol layering in Transmission Control Protocol. For the TCP layer to identify an application, a socket is bound to a port number. A socket is an endpoint of two-way communication between two applications connected to the network [14]. It is expected that the TCP can communicate asynchronously with the connected applications. A push function, which is used to send data to the receiver, is defined to guarantee that data submitted to the TCP is transmitted. The data bytes being sent are referred to as TCP segments, and they are sent as internet datagrams. Every message sent consists of a header, which represents the information at the beginning of the message. The TCP header consists of different sizes of data bytes that carry information about the TCP protocol such as a source port, 6.

(16) 3.3. Threading. 7. destination port, sequence number, and acknowledgment number. The sequence number is a number of the first byte of data in a segment. The acknowledgment number contains the value of the next sequence number that the client is trying to send to the receiver. The acknowledged number is always sent once a connection has been established [13]. The TCP header contains additional fields that specify where the data being sent begins in the segment and how to handle it. However, these details were considered too specific for the context of this report.. 3.2. Peer-To-Peer Networking. Peer-To-Peer (P2P) systems allow for efficient resource sharing and message routing between systems. Each connected computer or software is often referred to as a peer or node. The basic idea of a P2P system is to build the system as a virtual overlay on top of the physical network. An overlay allows peers to connect and disconnect from the same network, even if the physical network changes. The three main characteristics of a P2P system are self-organization, symmetrical communication, and distributed control. Self-organization signifies a computational grid that can adapt itself when a new peer is connected or disconnected. Symmetrical communication means that each node can either serve as a client or server. That the system has distributed control implies that there is no need for hierarchical order or authority to control the distributed resources. These three features allow a balanced, fault tolerance and scalable implementation of the resource sharing mechanism P2P [15].. 3.3. Threading. Parallel programming aims to create programs that can execute different parts of code tasks simultaneously. When independent pieces of a program can run parallel to each other, the response time is decreased and the performance of the program is enhanced. Concurrent programming is when concurrent systems conflict over the same Central Processing Unit (CPU) to run tasks. Any conflict is handled by a CPU scheduler that decides which task is apt to run at a specific time. The CPU scheduler can process these decisions so fast that it seems like code tasks are running simultaneously [16]. Thread programming on one CPU achieves the impression of code tasks running simultaneously while multi-threading on multiple CPUs achieves complete parallelization. A single thread is a sequence of programmed instructions that can run independently in a program. A single-threaded application is usually a heavyweight process with a single thread in control. In a multi-threaded application, the multiple threads do not have a progression order and can therefore execute concurrently with each other. Both single- and multi-threading are especially useful when tasks may block each other, and the program wants to proceed without waiting for specific tasks to finish [17]. For example, if two software wants to communicate, they should send messages while listening to incoming messages. Using multiple threads.

(17) 8. Chapter 3. Theory. for these actions ensures that the two software do not have to wait for the other before proceeding.. 3.4. Unicode. The Unicode Standard is used in programming for a universal encoding of composed characters and text [18]. The standard includes several associated specifications that help achieve interoperability between different implementations and communication between programs. Unicode and its specifications characterize a predictable method of encoding multilingual content that empowers the trading of text information universally and establishes worldwide programming. Unicode is required in new Internet protocols and implemented in all cutting edge frameworks and coding languages [18]. Unicode used the American Standard Code for Information Interchange (ASCII) character as its starting point. The Unicode Standard goes a long way past the defined capacity of ASCII to encode just the uppercase and lowercase letters A through Z. The ability to encode all characters utilized in the written languages of the world is given with Unicode. The character encoding treats alphabetic characters, ideographic characters, and symbols equally, which implies that they can be utilized in any combination. The Unicode Standard determines a numeric worth and a name for every one of its characters. In this regard, it is like other character encoding guidelines originating from ASCII [18].. 3.4.1. UTF-8. Unicode characters are expressed through three Unicode Transformation Formats (UTF): UTF-32, UTF-16 and UTF-8. The 8-bit form, UTF-8, is a byte-oriented form designed for usability with existing ASCII-based frameworks. UTF-8 includes the ASCII range of Unicode and thereby maintains an ASCII compatibility using 8-bit code units. Plenty of existing software rely upon character information being represented as a sequence of bytes. The most straightforward approach to adjusting Unicode usage to such a circumstance is to utilize an encoding structure defined in 8-bit code units. That is the purpose of UTF-8, and why it is preferable for text encoding [18]..

(18) 3.5. OpenSpace Architecture. 3.5. 9. OpenSpace Architecture. The OpenSpace architecture (see Figure 3.1) contains four layers; OpenSpace-Core, OpenSpace applications, external applications and modules. OpenSpace-Core is a software package that consists of different components to manage the rendering, define Application Programming Interface (API) support extensibility, and determine the overall application flow. The main application uses OpenSpace-Core, modules, and a selected windowing framework to create an executable file for the user to run. External applications are allowed to interact with OpenSpace via the definition of a network API. Modules in OpenSpace allow code customization and are self-contained packages that provide specialized instances of C++ classes. The C++ classes can be used to render objects, data management, and provide scripts to influence the rendering of objects defined in a module. In OpenSpace, scene graph nodes are used to organize all relationships of objects in the scene. All scene graph nodes have an unique identifier that is part of a unique identifier system to change the rendering parameters of the scene graph node. A renderable class in OpenSpace renders different objects in the scene, such as stars and planets. OpenSpace maintains one single scene graph that represents the parent/child relationships of all loaded objects. Furthermore, these objects are defined in different modules. With minimal knowledge about the rest of the system, additional functionality can also be added to OpenSpace via modules. Modules can also define new Lua functions that can be called from the command line console inside the application [19].. Figure (3.1): Simplified overview of the system architecture of OpenSpace. Image taken from [1]. OpenSpace uses an asset system to set up the scene content and settings. The Property System contains user-changeable settings referred to as properties. The values for properties can be changed via the GUI, asset system or through the scripting API. Scripting also allows the user to make changes to the scene graph structure during run time, such as changing render parameters. When a user opens OpenSpace, the current focus node is set to be Earth. When the user navigates around in the scene by rotating and zooming, the navigation occurs only in relation to the current focus node. The user can change the focus node, which will also affect the navigation in the scene [1]..

(19) Chapter 4 | Implementation. The investigation of implementation methods in this thesis led to the discovery of an existing messaging protocol for astronomy software called SAMP, described in Chapter 2. The exploration of SAMP led to it being considered a method early on in the project. A benefit of using SAMP would have been that it would not have been necessary to implement an independent messaging protocol for OpenSpace. SAMP is also commonly used by astronomy software, and integrating it with OpenSpace would have made OpenSpace compatible with a list of software [20]. Despite having several benefits, SAMP was not chosen as a method for this thesis. It was decided that implementing a messaging protocol from scratch would give OpenSpace more developer and functionality control. The advantages and disadvantages of using SAMP are discussed further in Appendix A, where the decision is justified. Even though SAMP was not used for communication, it greatly influenced how to create a messaging protocol for OpenSpace. The objective was to expand the proof of concept created to integrate Glue and OpenSpace and make it more resilient. This objective could be attained through several implementation goals.. 4.1. Implementation Goals. Implementation goals were set for further developing the proof of concept plugin to integrate Glue and OpenSpace. Glue represents future software integrations with OpenSpace. The implementation goals also serve as the basis of the messaging protocol for OpenSpace. For OpenSpace and Glue, it was considered necessary to change from the use of WebSockets to TCP-sockets. The functionality of the two sockets is conceptually similar, and the implementation as well. WebSockets are primarily used for communication between a server application and a browser-based application. Since Glue and most astronomy software are not browser-based, a direct TCP-socket connection was deemed more suitable. How messages are sent through the socket needed to be implemented as well. The goal was to send and receive messages as binary strings, with the intention of sending and receiving large data sets without delay. Furthermore, OpenSpace and Glue were not capable of both sending and receiving messages. Instead, Glue was capable of sending messages and OpenSpace receiving them. The objective was to enable twoway communication by using threading. Threading allows for several parts of the 10.

(20) 4.2. Messaging Protocol. 11. plugin to run concurrently as separate flows of execution. Threading thereby helps to maintain asynchronous communication. The main implementation goal for the communication with OpenSpace was to establish specific message types described in a messaging protocol. Knowing the structure of an incoming message simplifies how to parse the message and get the relevant information. A structured protocol contributes to an intuitive messaging development for software that wants to connect with OpenSpace. The content of the protocol decides and describes what actions are possible between OpenSpace and connected software. These actions include connection and disconnection to software, sending and receiving data, sending and receiving data properties, updating data properties, and removing data. Lastly, the messaging protocol requires a structure that makes Glue, as well as future integration software, self-reliant on the infrastructure of OpenSpace. Keeping the software functionalities separate makes neither software dependent on the other’s infrastructure. Thus, OpenSpace can update its infrastructure and functionality without Glue or other software updating its messaging to OpenSpace.. 4.2. Messaging Protocol. A messaging protocol for OpenSpace was created to establish an asynchronous messaging standard between OpenSpace and connected software. The messaging protocol works as a basis for the implementation of receiving and sending messages in OpenSpace. The implementation of this messaging protocol followed three key concepts: • Construct messages as strings that include a structured message sequence • Structure messages with a header that includes the protocol version and the message type • Structure messages to send the least number of bytes possible without losing data The first bytes of a message sequence indicates the number of bytes to follow in a multibyte sequence, which allows for efficient forward parsing. The goal was to send as few bytes in a message as possible and anticipate enough room for the data in its assigned byte size in the buffer sequence. Therefore, the first bytes of all messages were decided to be referred to as a header. The header is and must be included in all messages to and from OpenSpace. The first 16 bytes of a message defines the header, consisting of the protocol version, message type, and the total length of the subject. In this context, the subject is the content, or payload, of the message itself. The subject can represent various actions, such as sending newly added data sets or subsets, or updating existing data properties..

(21) 12. Chapter 4. Implementation. The message type determines what kind of subject is in the message. The different message types that were created are: Connect (CONN), Disconnect (DISC), Point Data (PDAT), Add Scene Graph Node (ASGN), Remove Scene Graph Node (RSGN), Update Color (UPCO), Update Opacity (UPOP), Update Size (UPSI), and Toggle Visibility (TOVI).. 4.2.1. Header. The header is the central part of all messages and is included regardless of the message type. The header is used to determine what kind of message has been received and how large the subject is. The arguments for the header are structured as followed: • Protocol Version (3 bytes, string) • Message Type (4 bytes, string) • Length of the subject (9 bytes, string) The protocol version is set to take up three bytes of the header because it is formatted as a number with one decimal. Every time the protocol is updated, the protocol version will increase by one decimal point. For example, if the first version is “1.0”, then the second version is expected to be “1.1”. If the version exceeds the number “9.9” in future development, then the protocol version will take up four bytes of the header as the next version is “10.0”. In the event of the messaging protocol for OpenSpace updating to a newer version, OpenSpace can keep the functionality for the previous protocol versions in its infrastructure. In this way, connecting software does not have to immediately update its messaging to OpenSpace to match the new protocol version. Instead, they can keep their messaging corresponding to a previous version and still communicate with OpenSpace. The message type was chosen to take up four bytes of the header. This number was chosen because four characters are enough to create an understandable acronym for a message type. More characters would take up unnecessary space and increase the size of the header. If new message types are added in the future, they will also have to have an acronym of four characters. The length of the subject takes up nine bytes of the header, even when subjects with small lengths are sent. Every message received by OpenSpace starts with the header, and the subject can therefore be anything from a new property size to hundreds of point data. The subject length needs a large enough predefined number of bytes in the header to avoid losing data. When OpenSpace receives a message, it expects that the first 16 bytes are the header and start reading those bytes. The first three bytes will always be the version number of the protocol. When those three bytes are read, OpenSpace saves the string value as an integer and compares it to the current protocol version in OpenSpace. If.

(22) 4.2. Messaging Protocol. 13. the protocol versions do not match, OpenSpace will throw an error and disconnect the connected software. If they match, OpenSpace will continue reading the next four bytes of the message and save them as a string representing the message type. The next nine bytes in the message represents the size of the message subject. Those bytes will be read and saved in a string that is converted to an integer. The saved integer determines how many bytes are left to read from the message. Those bytes represent the message subject and will be read and saved in a message buffer. Depending on the message type, the message buffer will be read accordingly.. 4.2.2. Connect (CONN). The message type CONN was created for sending a message that ensures a connection has been made between OpenSpace and software. After a software has connected to the specified port for OpenSpace, it should send an initial message that confirms the connection. The arguments of messages of this type are structured as followed: • Header (16 bytes, string) – Protocol version, message type (CONN) and length of the subject • Subject (x bytes, string) – Name of the software that connected to OpenSpace. 4.2.3. Disconnect (DISC). The DISC message type is sent in messages to disconnect the existing connection from the socket server. This message type is the only exception where the header is not sent in its regular format. No subject needs to be sent when disconnecting, so there is no length of a subject to include in the header. The arguments of messages of this type are structured as followed: • Protocol Version (3 bytes, string) – The version number of the messaging protocol • Message Type (4 bytes, string) – DISC. 4.2.4. Point Data (PDAT). The PDAT message type is used to provide point data to OpenSpace from a connected software. Every point in the data expects to have x-, y- and z-coordinates. For the point data to be used and visualized in OpenSpace, the scene graph node message type (see Section 4.2.5) must be sent after PDAT with the associated point properties. OpenSpace renders the point data received from PDAT in the scene and changes its focus node to the scene graph node created from ASGN. The arguments of messages of this type are structured as followed:.

(23) 14. Chapter 4. Implementation. • Header (16 bytes, string) – Protocol version, message type (PDAT) and length of the subject • Subject (x bytes, string): – Length of x-coordinates (9 bytes, string) - The total length of all the x-coordinates – X-coordinates (x bytes, string) - All the data points x-coordinates, given in meters – Length of y coordinates (9 bytes, string) - The total length of all the y-coordinates – Y-coordinates(x bytes, string) - All the data points y-coordinates, given in meters – Length of z coordinates (9 bytes, string) - The total length of all the z-coordinates – Z-coordinates (x bytes, string) - All the data points z-coordinates, given in meters. 4.2.5. Add Scene Graph Node (ASGN). The ASGN message type should be sent directly after a PDAT message type. ASGN is used to send essential information to OpenSpace to create a scene graph node of the point data received by PDAT. The necessary information is: a scene graph node identifier, a Graphical User Interface (GUI) name, and the color, opacity, and size of the data points. The properties needed to create a scene graph node in OpenSpace have been given a specified byte length. The arguments of messages of this type are structured as followed: • Header (16 bytes, string) – Protocol version, message type (ASGN) and length of the subject • Subject (x bytes, string): – Length of identifier (2 bytes, string) – The total length of the identifier for the scene graph node – Identifier (x bytes, string) – The identifier representing the scene graph node – Length of color (2 bytes, string) – The total length of the color value of the point data – Color (x bytes, string) – The color of the point data – Length of opacity (1 byte, string) – The total length of the opacity value of the point data – Opacity (x bytes, string) – The opacity value for the point data.

(24) 4.2. Messaging Protocol. 15. – Length of size (1 byte, string) – The total length of the size value of the point data – Size (1-3 bytes, string) – The size value of the point data – Length of GUI name (2 bytes, string) – The length of the given name of the point data – GUI name (x bytes, string) – The given name of the point data The identifier will represent the scene graph node created in OpenSpace and should therefore be unique. The identifier can be a randomly generated set of characters. The length of the identifier must be a double-digit number of characters so that the length takes up exactly two bytes in the message subject. This was decided because an identifier with a length between 0-9 characters was considered too short and that more characters would make the identifier more unique. There was no need to make the length take up three bytes in the message subject as a three-digit identifier was considered excessive. The length of the GUI name follows the same rules as the identifier. The GUI name should be descriptive of the point data it belongs to, so it is easy to navigate the OpenSpace GUI. The color value sent in a message subject is a three-dimensional RGB vector. The vector contains normalized values starting and ending with parentheses and a comma separating each of the values. For example, one possible color value could look like this: (0.65, 0.2, 0.785). This format was decided because with commas separating the red, blue, and green values, it would be easier for a program to separate them from each other. The length of the color was also decided to be a double-digit number and take up two bytes. The opacity and size value sent in a message subject was decided not to exceed a length of one digit. Opacity is a value between zero and one that can include many decimal points. Therefore, the opacity value can have a maximum of seven decimals so the value does not exceed a length of one digit. The size value sent in a message subject is any real number between 0-150 with no decimals and will not have a length greater than three.. 4.2.6. Remove Scene Graph Node (RSGN). The RSGN message type is used to remove scene graph nodes from the scene in OpenSpace. The reason can be when data is removed, or a subsets data is updated to new points. If a the data points of a subset are updated, then the current scene graph node for that subset should be removed, and a new scene graph node should be created from the new data points. The arguments of messages of this type are structured as followed:.

(25) 16. Chapter 4. Implementation. • Header (16 bytes, string) – Protocol version, message type (RSGN) and length of the subject • Subject (x bytes, string): – Identifier (x bytes, string) – The identifier representing the scene graph node to be removed Note that the length of the identifier is not sent in the subject of RSGN. Since the identifier is the whole subject, then the length of the subject represents the length of the identifier.. 4.2.7. Update Color (UPCO). A UPCO message is sent when the color property of a scene graph node has been changed in one software and needs to be communicated to the other connected software. The arguments of messages of this type are structured as followed: • Header (16 bytes, string) – Protocol version, message type (UPCO) and length of the subject • Subject (x bytes, string): – Length of identifier (2 bytes, string) – The total length of the identifier for the scene graph node – Identifier (x bytes, string) – The identifier representing the scene graph node – Length of color (2 bytes, string) – The total length of the color value of the point data – Color (x bytes, string) – The color of the point data. 4.2.8. Update Opacity (UPOP). A UPOP message is sent when the opacity property of a scene graph node has been changed in one software and needs to be communicated to the other connected software. The arguments of messages of this type are structured as followed: • Header (16 bytes, string) – Protocol version, message type (UPOP) and length of the subject • Subject (x bytes, string): – Length of identifier (2 bytes, string) – The total length of the identifier for the scene graph node.

(26) 4.2. Messaging Protocol. 17. – Identifier (x bytes, string) – The identifier representing the scene graph node – Length of opacity (1 byte, string) – The total length of the opacity value of the point data – Opacity (x bytes, string) – The opacity value for the point data. 4.2.9. Update Size (UPSI). A UPSI message is sent when the size property of a scene graph node has been changed in one software and needs to be communicated to the other connected software. The arguments of messages of this type are structured as followed: • Header (16 bytes, string) – Protocol version, message type (UPSI) and length of the subject • Subject (x bytes, string): – Length of identifier (2 bytes, string) – The total length of the identifier for the scene graph node – Identifier (x bytes, string) – The identifier representing the scene graph node – Length of size (1 byte, string) – The total length of the size value of the point data – Size (1-3 bytes, string) – The size value of the point data. 4.2.10. Toggle Visibility (TOVI). A TOVI message is sent when the visibility property of a scene graph node has been changed in one software and needs to be communicated to the other connected software. The arguments of messages of this type are structured as followed: • Header (16 bytes, string) – Protocol version, message type (TOVI) and length of the subject • Subject (x bytes, string): – Length of identifier (2 bytes, string) – The total length of the identifier for the scene graph node – Identifier (x bytes, string) – The identifier representing the scene graph node – Value (1 byte, string) – Must be sent as “T” representing true or as “F” representing false Note that the length of the value is not sent in this message type. This is because it will always be one, and there is no need to send the extra byte..

(27) 18. 4.3. Chapter 4. Implementation. Receiving Messages in OpenSpace. A connection protocol is needed to make communication possible between OpenSpace and other astronomical software. The internet protocol used for this thesis was the Transmission Control Protocol, which provides reliable communication between applications connected to a network. P2P networking was the choice of implementation for the communication network architecture, allowing resources to be shared between applications. The applications are often referred to as nodes when it comes to P2P networking. In this case, the nodes are OpenSpace and the connected application. OpenSpace starts the connection and listens for clients trying to connect to the socket. The socket is only alive during the lifetime of the OpenSpace application running. When a connection has been established via the local address, a thread is initialized to stream data. The socket listens for incoming messages on a specified port, where these messages are referred to as peers. If there is an incoming peer, the socket starts to receive the content of that peer message being sent from another application. The receiving of data is restricted to the given address; messages from other addresses are discarded. The process of receiving data from a connected socket includes; a descriptor that identifies the connected socket, a buffer to receive the incoming data, and the length in bytes of the buffer. A buffer size specifies the maximum amount of data accepted. If the data exceeds the specified size of the buffer, the buffer will be filled with the first part of the total amount of data and then block the execution until enough data has entered the input queue. If there is no incoming data at the socket, the call for receiving data is blocked, and it will wait for data to arrive. Depending on the peer message content, the message is distributed to different cases to parse the data accordingly. The different cases are the message types described in Section 4.2.. 4.3.1. Rendering Received Point Data in OpenSpace. Received point data from the PDAT message type described in Section 4.2.4 is rendered with OpenGL as circular points in OpenSpace. The points are rendered using double precision since merely floating point precision would not give sufficient precision the vast distance in space. Each data point in a data set are characterized by x-, y- and z-coordinates, given in meters. These coordinates are used in OpenSpace to determine the three-dimensional space position of a data point. In OpenSpace, astronomical objects positions are given in meters and multiplied by a parsec (pc) constant. A parsec is a length unit used to measure distances to astronomical objects, stars and galaxies, outside the Solar system. One parsec is approximately equal to 31 trillion kilometers [21]. Since the coordinates for the data points are given in meters, the data points will not have the correct position in OpenSpace without the parsec constant multiplied with each data points position. With the correct length unit, each data point is rendered with the associated properties, color, size and opacity, received from the ASGN message type..

(28) 4.5. Testing the Protocol with Glue. 4.4. 19. Sending Messages from OpenSpace. In addition to receiving messages from clients, OpenSpace can also send messages. This is done in the case of a property change to data in the active renderable. A user can change the value of properties in the OpenSpace GUI. Changeable properties include the color, size, opacity, and visibility of the rendered points. OpenSpace uses a callback function to check if a property value has changed. It works as a subscription to a property value, which can be accessed by specifying its unique identifier. Whenever the property value updates, the client receives a message specifying the new value of that property. The message sent follows the same messaging structure that OpenSpace receives from a client.. 4.5. Testing the Protocol with Glue. The experimental plugin for Glue and OpenSpace was further developed side by side with the messaging protocol for OpenSpace. The plugin was developed to be able to test the implementations made in OpenSpace for the messaging protocol.. 4.5.1. Sending Messages. The first step was changing the use of sockets from WebSockets to TCP-sockets. The previous plugin connected to the WebSocket for OpenSpace using the “websocket” library for Python. Instead, the plugin connection changed to the use of the “socket” library for Python. Changing the socket connection in the Glue-OpenSpace plugin did not affect the relation between OpenSpace and the plugin since it was still sending Lua scripts as messages. As mentioned in Section 2.2, to send messages from Glue to OpenSpace, Glue sends Lua script commands as messages. The goal was to change the messages to binary strings. Changing to binary strings meant updating the messaging from Glue and the content structure of those messages. This change included updating how a message is sent when data in Glue is updated or added, subsets created, and scene graph nodes removed. In the previous plugin messages were sent in JSON format (JavaScript Object Notation) encoded in ASCII. Messages were sent in JSON format to create a JSON array that aligned with the message structure required for calling a Lua script. By changing the message content from JSON arrays to strings, there is no need to send messages in JSON format. Instead, the message can be directly converted from a string to bytes encoded in UTF-8. Without a character conversion like UTF-8, the computer cannot store the string messages sent by the plugin. This issue would result in the messages being lost in translation and could not be read by the receiving end, OpenSpace..

(29) 20. 4.5.2. Chapter 4. Implementation. Sending Data. The data transmission was changed in the Glue-OpenSpace plugin to match the new messaging standard. As mentioned in Section 2.2, the plugin previously created a temporary file on the computer and then, in a message, told OpenSpace the location of that file. This method was changed to creating strings of the data, ordered as specified in the messaging protocol for OpenSpace. Data is sent from Glue when a connection is made with OpenSpace and when regions of interest are created in Glue. Regions of interest correspond to subsets of data highlighted in Glue and sent to OpenSpace. Changing how messages are sent from Glue led to the content of the messages being restructured, as described in Section 4.2. This structure included creating strings that contain information in a sequence that OpenSpace expects to read the information. The calls for scripts and functions are made entirely in OpenSpace after a received message has been read. OpenSpace functionality is kept in OpenSpace and Glue functionality in Glue by not calling for Lua scripts in the plugin messages. When loading a data set into Glue, the longitude and latitude values need to be specified. For data sets to be sent to OpenSpace, Right Ascension (RA) and Declination (DEC) is used. RA is equivalent of longitude on an abstract celestial sphere. The celestial sphere is useful for describing locations of objects in the sky. DEC is equivalent of latitude on the same celestial sphere. An illustration of the celestial sphere can be seen in Figure 4.1. DEC is expressed in degrees, RA can be expressed in degrees as well but is more common to specify in time [22]. The RA and DEC values together with the average standard distance for a data point is converted to cartesian coordinates before being sent to OpenSpace. The conversion is done by using the Python library Astropy and its class called SkyCoord. The class provides flexible celestial coordinate representation, manipulation and transformation between systems [23].. Figure (4.1): Celestial sphere illustration [24]..

(30) 4.6. User Tests. 4.5.3. 21. Receiving Messages. Since OpenSpace can send messages, the Glue-OpenSpace plugin had to be updated to receive messages. The structure of these incoming messages is in the same structure that Glue sends messages to OpenSpace. Incoming messages contain strings with information in a specified sequence declared by the messaging protocol for OpenSpace. The incoming messages from OpenSpace consist of property updates to scene graph nodes and point data. Making Glue receptible to these incoming messages indicated adding a receive message function to the socket. The plugin needs to continue listening for incoming messages as long as Glue is connected to OpenSpace. Threading was used to accomplish active listening for incoming messages. A thread instance was created for listening to incoming messages on a separate thread. The thread is then told to run a separate function that checks if there is a socket connection and runs a second function to read the incoming message. An incoming message is read by saving the data of the socket receive buffer to a local string. That string is then handled differently depending on the message type it includes. For example, if the message type received is UPCO, the message contains a color value, and the color of the data points will be updated in OpenSpace.. 4.6. User Tests. User tests were carried out to evaluate the messaging protocol created for OpenSpace and the further developed Glue-OpenSpace plugin. The user tests aimed to help find flaws in the message communication between OpenSpace and Glue. The main focus was users who had familiarized themself with the old plugin. An instructional guide (see Appendix C) and a questionnaire (see Appendix D) were sent out to a selected group of users. These users included an OpenSpace developer, a visualization researcher, and an astronomer. The instructional guide introduced the users to how to get started with installing both Glue and OpenSpace. The guide then instructed the user on how to set up the plugin and connect Glue to OpenSpace. After the instructional tasks are done, the user was given a list of user-based tasks to perform in Glue and OpenSpace. The user was encouraged to make notes of any irregularities or communication flaws during the user test. These notes could then be entered into the questionnaire after the user test was completed..

(31) Chapter 5 | Result. This thesis aimed to integrate other astronomical software with OpenSpace, which was done by creating a messaging protocol called Software Integration Messaging Protocol (SIMP) for OpenSpace. The protocol, presented in full in Appendix B, describes a structure for how messages should be sent and received. The resulting messaging protocol enables the communication between OpenSpace and astronomy software tools. Simple use cases are demonstrated below to vizualise the result. The protocol can be used for rendering point data in OpenSpace. The data set used in this presentation of the result is a data set on exoplanet systems, where the position is matching the position of the host star in the system. The size of the used data set is four megabytes (MB). This data set is only used as an example, other data sets can be used as long as the cartesian coordinates for the points can be derived from the data. The protocol consists of nine different message types that trigger various actions in either OpenSpace or the connected software. The existing message types are: • CONN - Connection • DISC - Disconnection • PDAT - Point Data • ASGN - Add Scene Graph Node • RSGN - Remove Scene Graph Node • UPCO - Update Color • UPOP - Update Opacity • UPSI - Update Size • TOVI - Toggle Visibility. 22.

(32) 23. In the use case shown in Figure 5.1, Glue has visualized the exoplanet system data through a scatter plot. Once a connection is made via Glue and OpenSpace, the data is visualized in a 3D immersive environment in OpenSpace. The x-axis of the scatter plot in Glue is set to the measure Right Ascension (RA), and the y-axis is set to Declination (DEC) of each star. The data visualized in Figure 5.1 and Figure 5.2 shows the spread of exoplanets in the universe.. Figure (5.1): The interface of Glue when a connection has been made with OpenSpace. The data points are visualized in a scatter plot. The x-axis represents RA and the y-axis represents DEC. The data set is visualized in both OpenSpace and Glue..

(33) 24. Chapter 5. Result. Figure (5.2): The interface of OpenSpace when a connection has been made with Glue. The visualized data set matches the one presented in Figure 5.1. The view in OpenSpace is the view of the Sun from Earth, where the bright white light is the Sun. The user can add subsets and make changes to the color, size, and opacity of the data points. The result of creating two subsets in Glue with synchronized visualization in OpenSpace can be seen in Figure 5.3 and Figure 5.4..

(34) 25. Figure (5.3): Visualization of two subsets added in Glue over the original data set shown in purple. Where the first subset is the data points colored in cyan. The second subset is the data points colored in orange.. Figure (5.4): The two subsets added in Glue, visualized in OpenSpace. The visualization corresponds to Figure 5.3. The second subset in orange is not as visible as it is in Glue. This visual impairment is caused by the second subset overlaying the original data points..

(35) 26. Chapter 5. Result. Figure 5.5 shows another use case, a scatter plot view with a belonging table view. The number of planets and stellar mass are set as x- and y-axis. Figure 5.5 and Figure 5.6 a chosen subset of the exoplanet data where the systems with a certain number of planets are highlighted.. Figure (5.5): Visualization of exoplanet data where the systems with a certain number of planets are highlighted in Glue. The x-axis shows the number of planets and the y-axis shows the mass of planets. A subset of the data set on exoplanet systems is selected in yellow. The subset selected is a system containing five planets..

(36) 27. Figure (5.6): Visualization of exoplanet data where the systems with a certain number of planets are highlighted in OpenSpace. The visualization correspond to Figure 5.5. The user is allowed to change properties on the data set in the respective menu in OpenSpace and Glue, see Figure 5.7 and Figure 5.8. When a change is made in one software, the update is synchronized to the other software through message communication. The available parameters in both GUI that will synchronize between the two software are: • Change in color of the points in a data set • Change in opacity of the points in a data set • Change in size of the points in a data set • Change in visibility of the points in a data set.

(37) 28. Chapter 5. Result. Figure (5.7): Menu for property change on point data in OpenSpace. The properties related to this master thesis project are Color, Opacity, Size and Toggle Visibility.. Figure (5.8): Menu for property change on data points in Glue. The slider under color is used to set the opacity value. Under the tab "Size", the size of data points can be changed by entering a number..

(38) 5.1. User Tests. 5.1. 29. User Tests. The user tests consisted of three user tests done by an astronomer, a visualization researcher, and an OpenSpace developer. The result of their separate user tests have been combined to find a pattern in user difficulties. Therefore, some of the answers are presented as the average results from user gradings in the range 1-5. Averages and responses below 2.5 are described as not difficult. Averages and responses at 2.5 are described as somewhat difficult. Averages and responses above 2.5 are described as difficult. The users found getting started with the plugin to be somewhat difficult, and the user experience in Glue seemed a bit worse than in OpenSpace. For the questions regarding the user experience in Glue, the result showed an average below 2.5. When it came to changing the properties of color and opacity in Glue, the users found it not difficult on average. Changing the size property and adding subsets in Glue showed to be somewhat difficult for two users and not difficult for one user. Two of the users also found it harder to change properties of a data set versus a subset. The interactions in OpenSpace were all on average considered not difficult by the users. In OpenSpace, none of the users found any difference in difficulty to change the properties of a data set versus a subset. One user noted that changing the color property in OpenSpace is not more challenging but more inconvenient in OpenSpace than in Glue. In Glue, the color is altered through a dropdown menu with visual colors, but in OpenSpace, you have to set a number for the separate RGB channels. Therefore, the user found it more user-friendly to change the color property in Glue rather than in OpenSpace. Two of the users had tried the previous plugin for Glue and OpenSpace. One of those users found the new plugin to have a smoother and more robust interaction. The other user noted that the new plugin was an improvement from the old plugin. However, two of the three users experienced communications flaws during their user test that affected the user experience of the new plugin. Two users also experienced a delay of 2-5 seconds sometime during the user test. Only one user ran into synchronization issues and managed to get both Glue and OpenSpace to crash during their user test. That user also tried to reach the data size limit of the plugin to make the plugin crash but was unsuccessful..

(39) Chapter 6 | Discussion. The user test results presented in Section 5.1 implies that improvements have been made compared to the previous plugin. The resulting messaging protocol is a step in the right direction to make OpenSpace interoperable with astronomy software. However, there is room for improvement. There are still some missing features that could be part of the future development of the messaging protocol.. 6.1. Implementation. The message protocol created is adapted to render data points and the data properties position, color, size, opacity and visibility. The protocol could have been adapted to several renderables and different kinds of data properties with some extra time. Furthermore, if given more time, an early prototype could have been completed and tested by the user. Contact was made with users about the implementation during the project, but there was only one user test of the final protocol. Several suggestions could have been received earlier on to improve the protocol. Awareness of missing features or bugs could have been discovered and taken care of by having a prototype tested earlier in the project. The user tests could also have been improved by adding questions to the questionnaire, more participants and as mentioned perform user tests at an earlier stage. The questionnaire could have been improved by asking the user to specify which data set and the size of the data set used. In that case we could have presented numbers of the size of the data sets used and perhaps draw conclusions to issues that occurred during the user tests. This would have also allowed us to present a specified number of bytes that is proven for the message protocol to handle. The result of the questionnaire would most likely be improved by letting more users participate. The cause of why more users did not test the protocol, is due to the amount of work needed to get both OpenSpace and Glue running if the user does not have either of them installed. However, we could have executed one more user test at an earlier stage. By having a user test at an earlier stage, the protocol could have been developed further with more focus on the needs of the user, and unknown issues could have been brought to our attention. At the beginning of the implementation of the protocol, an executable version of the protocol existed where point data was accessed via local temporary files, the same method used by the old plugin described in Section 2.2. The sending of point data had not yet been improved and the PDAT message type was not implemented. User tests could have been performed at this 30.

(40) 6.1. Implementation. 31. stage to only test the communication between Glue and OpenSpace with the other message types implemented. The results from those user tests could then have been compared to the results of the last user tests when the PDAT message type was implemented. This could have proven a distinctive difference in the protocol before and after point data was sent as binary strings. During the user tests it sometimes occurred network related issues, such as the connection made was unstable or to the extent that a connection could not be created at all between Glue and OpenSpace. The cause of the problem was hard to locate due to not knowing if it was related to OpenSpace, Glue or perhaps both. During the development of the messaging protocol, and during one of the user tests as well, both Glue and OpenSpace froze. Speculations were made that this too could be a network related issue, or perhaps that the size of the data sets used in the user tests had reached maximum size for the protocol to handle. However, both of these issues, the network related issue and maximum size of the data set, could have been examined further. A more detailed debugging session would be needed for the network related issue and different sizes of data sets should have been tested. The infrastructure of the protocol is implemented to receive and render point data specifically. The implementation of the protocol in OpenSpace, may be expanded with functionality of other rendering techniques than the rendering of point data. Other rendering techniques were not explored in this project, because the rendering of data was not the primary focus of this thesis. The importance of the project is indeed the communication between software. The techniques used, Socket, P2P, and TCP, were already implemented in OpenSpace and convenient to use. Whether other methods would improve the result is possible but not tested. However, it could have been explored further at the beginning of the thesis project, what affect other techniques would have for the implementation. Other options for sending messages could have been User Datagram Protocol (UDP), Internet Control Message Protocol (ICMP) or Stream Control Transmission Protocol (SCTP), which all are similar to the Transmission Control Protocol. Nevertheless, the disadvantage of using UDP instead of TCP is that error messages are not considered. Messages would continue being sent, whether they are received or not. While the ICMP is typically used for sending error messages, it does not correspond entirely to what is desired to achieve with the protocol developed in this thesis. The lastly mentioned technique SCTP combines UDP and TCP and provides new functionality compared to the TCP, where the reliability of the data transfer is improved. This could be useful for further improvement of the protocol in cases where the user would like to send classified data. Since this was not a specified requirement, it was not taken into consideration for this master thesis project. By using other methods that not already exist in OpenSpace, it would have been necessary to implement them as well. This could have been quite time consuming and would perhaps be a master thesis itself. During the implementation phase, the project received one data set to use for testing the communication between Glue and OpenSpace. It was deemed not necessary to have multiple data sets to work with as it was only used to make sure data could be sent to OpenSpace. Therefore, the data set described in Chapter 5 was used for both testing the protocol and the implemented functionality. Had multiple data sets been used during the development, more extensive testing of the protocol would.

No results found