Development of an Interactive VR Experience for an Art Museum

(1)

Development of an Interactive VR Experience for an Art Museum

Max Enros

Computer Game Programming, bachelor's level 2020

Luleå University of Technology

Department of Computer Science, Electrical and Space Engineering

(2)

Abstract

In this project, a prototype for a virtual reality museum experience was developed.

The virtual environment contains paintings and birds that the user can interact with to display information. A text panel or an image is displayed next to the object the user interacted with. The text or image displayed contains information about the object. A narration of the information can be played for some of the images and text panels. This enhances the experience of the museum compared to looking at an object in the real world. The virtual reality experience allows the museum to be more interactive and fun for the visitors. A few user tests were performed to test the usability of the experience. The tests found a few usability problems that have to be fixed before this experience could be part of a real museum.

Sammanfattning

I detta projekt utvecklades en prototyp för ett virtual reality museum. Den virtuella miljön inneh˚aller tavlor och f˚aglar som användaren kan interagera med för att visa information. En textpanel eller en bild visas bredvid objektet som användaren interagerade med. Texten eller bilden som visas inneh˚aller information om objektet. En inspelning av informationen kan spelas upp för n˚agra av bilderna och textpanelerna. Detta förbättrar museets upplevelse jämfört med att titta p˚a ett objekt i den verkliga världen. Virtual reality-upplevelsen gör att museet kan vara mer interaktivt och roligt för besökarna. N˚agra användartester genomfördes för att testa upplevelsens användbarhet. Testerna hittade n˚agra användbarhetsproblem som m˚aste ˚atgärdas innan denna upplevelse kan vara en del av ett verkligt museum.

(3)

1 Introduction

Visiting an art museum with paintings is usually a non-interactive experience.

Mostly the visitors can only look at paintings and read about them. Virtual reality (VR) can be used to make the experience more interesting. A virtual environment containing interactive paintings can be used to add more interactive elements to the experience and making it more enjoyable [1]. When a user interacts with a painting in the virtual environment, objects, an image or, a text panel appears near the painting. The objects are either objects in the painting or objects relevant to the painting in some way. Information about the painting or information about the artist is displayed as a floating text panel next to the painting. A sound clip containing a narration can be played for some of the text and images. The use of audio and narrations in a virtual environment can create a more immersive experience for the users [2].

The virtual environment for this project is a square room without doors or windows. A painting of a child feeding sparrows hangs on one of the walls in the room. A picture of the artist hangs on the opposite wall. When the painting with the sparrows is interacted with, three birds fly out of the painting. Each bird can be interacted with to display an image of a bird or a text panel next to the painting. When the picture of the artist is interacted with, text panels appears containing information about the artist.

1.1 Background

Virtual reality is used to improve the experience of the museum by making it more interactive. Instead of just looking at paintings and reading texts, a VR environment can add new and fun ways to experience the museum. A virtual environment has almost endless possibilities for the user to interact with the museum in ways not possible in the real world. Because of this, the user will be more involved in the museum experience and might get more enjoyment out of it.

The paintings in the virtual environment are from the Anna Nordlander Mu- seum [3], which is a section of Skellefte˚a Museum [4]. Anna Nordlander [5] was an artist from Skellefte˚a that created the paintings. A couple of letters sent by her and information about her life is used as the text for the text panels to enrich the experience of the paintings. People from the museum created a storyboard containing their vision for the functionality of the virtual museum. This storyboard also contained a suggestion for the design of the text panels and images displayed next to the paintings.

(5)

2 Methods

This section describes the tools and devices used in this project, as well as how the user tests were performed.

2.1 Unity

The Unity [6] game engine was used to create the virtual experience. Unity can be used to create both 2D and 3D projects. Support for several types of VR equipment is available in Unity. The project can be compiled and run inside of Unity as well as uploaded directly to a VR headset for testing. These factors make testing quick and easy.

2.1.1 Editor

Figure 1: A picture of the Unity editor. The scene hierarchy is visible to the left, the preview of the scene is in the center, and the property inspector is visible to the right.

Unity has a scene editor where game objects can be added, removed, or modified in the scene. The editor has a scene hierarchy were all game objects in the scene can be found. The scene hierarchy is visible to the left in figure 1. Game objects can be attached as child objects to other game objects to group objects together. The properties of an object are displayed in the property inspector when an object is selected. The property inspector is visible to the right in figure 1. The properties can be changed to modify the object. The main part of the scene editor is a preview of the actual 3D scene, where all objects currently in the scene are visible.

This is shown in the center of figure 1.

(6)

2.1.2 Components

C# scripts attached to game objects are called components. Components usually have the methods Start and Update. The Start method is called when the object is created and handles the startup code of the component. The Update method is called every frame and is used when operations need to be performed constantly.

All public member variables of a component are displayed in the property inspector when the game object the component is attached to is selected. The public members can easily be modified in the property inspector to give specific values to different objects in the scene. Unity has many built-in components for commonly used things like drawing models and simulating physics on objects.

2.1.3 Prefabs

Game objects can be saved as prefabs to make it easier to work with multiples of the same object. A prefab contains the game object and all its components as well as all child objects and their components. All values that have been modified in the property inspector are also saved in the prefab. Multiple instances of the same prefab can be added to a scene. If the prefab is modified, all instances of the prefab will receive the same modifications. This makes it easy to manage multiple identical objects in a scene. Each instance of the prefab can be modified in the scene without affecting the others if small tweaks are needed.

2.1.4 Canvas

A canvas is a type of object in Unity used to display 2D information such as text and images. A canvas is usually used when creating a GUI. The default type of canvas will display its contents directly on the screen regardless of where the camera is pointing. The canvas type can be changed to a world space canvas to have the canvas located in the actual world. This is useful for VR projects where the GUI will most likely be part of the 3D world.

2.2 Oculus Go

The Oculus Go [7] is a portable all-in-one VR headset that was used for this project.

The headset does not have to be plugged in to a computer to be used. This means that the headset can be used anywhere, and no cables will get in the way when using it. The Oculus Go has built-in speakers that can play sounds without the need for external headphones. A controller is included with the headset that the user holds in one of their hands. This controller is mostly used as a pointer for selecting things and navigating menus. A feature for recording what is displayed in the Oculus Go can be found in the headset. This feature was used to collect extra data while performing tests of the virtual experience.

(7)

2.3 Testing

To find out how easy this virtual museum experience was to use, usability tests where performed. Three test subjects volunteered to test the virtual experience and give feedback about the usability. Nothing was explained about the virtual experience to the testers, they had to figure out everything on their own. If they ever got stuck or did not notice something, they were helped to let them experience everything in the virtual museum.

2.3.1 Questionnaire

A questionnaire was created that the testers filled in after testing the experience.

The questionnaire was based on the System Usability Scale [10]. Three demographic questions were added at the beginning of the questionnaire. The rest of the questions have five options ranging from strongly disagree to strongly agree.

These options are numbered from one to five. See appendix 1 for the full questionnaire.

2.3.2 Interviews

In addition to the questionnaire, an interview was held with each tester about the experience. The point of the interviews was to discover what parts of the experience the testers found difficult to use or understand. The testers were also asked about suggestions for improvements or changes that would enhance the experience. These where the main questions asked:

Did you find anyhting difficult to use?

Did you find anything difficult to understand?

Do you have any suggestions for improvements to the usability?

Is there anyhting else you think should be changed?

The interviews were able to give more quality feedback and specific information than the questionnaire.

2.3.3 Observations

The testers were observed while using the virtual museum to see how easy it looked for them to use the VR headset. The recording feature of the Oculus Go was used to record each test. The recordings could later be analyzed to see how quickly and efficiently the testers were able to use the different features of the virtual museum.

(8)

3 Design and Implementation

This section describes the base functionality of interactive objects, as well as how the paintings and birds are created. The design of the GUI (Graphical User Inter- face) used by the paintings is also explained.

3.1 Database

A simple database was used to store the information displayed next to the paintings. The text and title for the text panels, as well as the images to display, are stored in this database. If a narration can be played for the image or text, an audio clip is also part of the information in the database. To keep track of which side of the painting the information should be displayed on, an extra value was added to the text panel and image information.

The database was used to allow interactive objects to load the information to display. If the information is modified in the database, the project will not need to be modified to display the updated information. For this project, an actual database was not used. Instead, all the information was hardcoded. The database structure was, however, designed in a way that allows a real database to be added later without affecting the rest of the project.

3.2 Interactive Objects

Objects in the scene that the user can interact with have an InteractiveObject component attached to them. This component handles all interactions the user can perform on the object. The database key for the information to display next to an interactive object can be set in the property inspector. All objects in this project will be paintings and birds, but more types of objects can easily be added later if needed.

3.2.1 Interacting with Objects

A component called Interactive was created to handle interactions. This component is used to handle the interactions made to an object. The InteractiveObject component inherits from the Interactive component to handle interactions. The object the Oculus Go pointer is pointing at is found using a raycasting function in Unity. A physics collider has to be attached to each interactive object for the raycasting to work. For this project, a box collider was added to all objects and was scaled to match the size of the objects. The functionality of the Oculus Go was implemented by another developer, but this specific part had to be connected to the interactive objects.

To make it more noticeable that an object is interactable, a glowing effect is displayed around the object while the Oculus Go pointer is pointing at the object.

(9)

The Interactive component or any component inheriting from it is responsible for highlighting the object.

3.2.2 Child Objects

The objects that fly out of a painting are child objects to the painting. Child objects can easily be added and removed from the parent object by using the property inspector. When an object is interacted with, all of its child objects will be activated. In the case of birds, activating means that the birds will fly out of the painting.

3.2.3 Paintings

A model of a painting was used for the paintings in the scene. This model consists of a frame surrounding an image of the painting. The image displayed in the frame can easily be changed by setting the texture of the painting model to the new image.

A painting prefab was created to make the work with paintings easier. This was achieved by first adding the painting model to the scene. An InteractiveObject component was then attached to the painting. This painting was then saved as a prefab. This prefab is shown in figure 2. When new paintings were added to the scene, the painting prefab was used. Every time the painting prefab is modified, all paintings in the scene will also receive the update.

Figure 2: The prefab of the painting.

(10)

Figure 3: A picture of the bird models. The male sparrow is to the left, and the female sparrow is to the right.

3.2.4 Birds

Like the paintings, the birds were created from models as well. Two bird models were used in this project. The first model was a male sparrow, and the second model was a female sparrow. Figure 3 shows the male sparrow to the left and the female sparrow to the right. An InteractiveObject component was attached to each bird to make the birds interactive.

One of the paintings in the scene has three sparrows as child objects. Two male sparrows and one female sparrow were added to the scene. The birds were then added as child objects to the painting. The birds will automatically start to fly once the painting is interacted with. The birds were created by another developer, but their flying was connected to the InteractiveObject component.

3.3 Graphical User Interface (GUI)

When an interactable object is selected, a GUI is displayed around the object containing more information about the selected object. The GUI consists of two parts. One part of the GUI is displayed to the left of the object, and the other part is displayed to the right. Each part of the GUI is capable of displaying either an image or a text panel.

3.3.1 Canvas

A world space canvas was used to display each part of the GUI. Since both the left and right side of the GUI for all interactive objects are identical, a canvas prefab was created. A world space canvas was added to the scene and then resized to a square a bit larger than a painting. This canvas was then saved as a prefab. The canvas is shown in figure 4.

A component called InteractiveObjectCanvas was created and attached to the canvas to make it easy to display an image or a text panel on the canvas. This component is used to both show and hide information on the canvas.

(11)

Figure 4: A world space canvas next to a painting. Visible as white lines.

3.3.2 Text Panel

The text panel used in the GUI contains a title and text on a rectangular background. This text panel was added to the canvas in the canvas prefab and can be seen in figure 5. The InteractiveObjectCanvas component can change the text and title of the text panel by using the information from the database.

Figure 5: An image of the text panel in the canvas prefab.

3.3.3 Images

Images are displayed in the GUI using a raw image [8] UI object. A raw image was added to the canvas of the canvas prefab and resized to a square. The image is visible in figure 6. The InteractiveObjectCanvas can change the image displayed to the information stored in the database.

In case the image is not a square, the RawImage has to be resized to preserve the aspect ratio of the image. If the width of the image is greater than the height, a ratio is computed by dividing the height by the width. The width of the RawImage is then set to the original side length of the square image. The height is set to the original side length multiplied by the ratio. If the height of the image is greater than the width, the same process is performed, but width and height are swapped.

(12)

Figure 6: A raw image added to the canvas prefab.

3.3.4 Audio

To be able to play audio, an AudioSource [9] component was attached to the canvas in the canvas prefab. If a database entry contains audio, the audio clip in the database is assigned to the AudioSource.

Buttons for playing audio, pausing audio, and closing the GUI were created to give the user more control over the audio. These buttons were added to the canvas as images, shown in figure 7. A new component that inherits from the Interactive component was created to allow the Oculus Go pointer to interact with the buttons.

Figure 7: A picture of audio buttons and an audio source added to the canvas prefab.

(13)

3.3.5 Interactive Object GUI

A component called InteractiveObjectGUI was created to let an interactive object control the GUI. This component can take a database entry and send the information to be displayed to the left and right canvas.

A prefab for the GUI was created to attach the GUI to interactive objects.

This prefab contains an empty object with the InteractiveObjectGUI component.

Two canvas prefabs were added to the empty object with a gap between reserved for an interactive object. This prefab is shown in figure 8. The GUI prefab was then added to the painting prefab to give the paintings a GUI, shown in figure 9.

When the birds are interacted with, the GUI attached to the parent painting is used to display the information.

Figure 8: A picture of the GUI prefab containing two instances of the canvas prefab.

Figure 9: A picture of the painting prefab containing an instance of the GUI prefab.

(14)

4 Result

This section presents the final prototype of the virtual museum experience. The results of each part of the tests are presented as well.

4.1 Virtual Museum

All of the planned features of the virtual museum were implemented. Nothing was left unfinished. The final prototype consisted of a square room with two paintings hanging on opposite walls. One of the paintings can be seen in figure 10. The GUI for interactive objects worked as intended and could display the information stored in the database. A painting GUI is shown displaying an image in figure 11 and is showing a text panel in figure 12. All three birds were activated when their parent painting was interacted with. Figure 13 shows two birds sitting on a table after being activated.

Figure 10: A painting hanging on one of the walls.

Figure 11: A painting GUI displaying an image of a female sparrow.

(15)

Figure 12: A painting GUI displaying a text panel full of information.

Figure 13: Two birds sitting on a table after being activated.

4.2 Tests

Three people participated in the testing of the virtual experience. The results of the tests are presented in this section.

4.2.1 Questionnaire

The results of the demographic questions are presented in table 1, and the scores of the usability questions are presented in table 2.

(16)

Question User 1 User 2 User 3

What is your gender? male female male

How old are you? 20 44 45

How experienced are you with virtual reality?

beginner beginner beginner

Table 1: The answers to the demographic questions.

Question User 1 User 2 User 3

I think that I would like to use this virtual museum

as part of a real museum 4 4 5

I found the virtual museum unnecessarily complex 2 2 3 I thought the virtual museum was easy to use 3 4 5 I think that I would need the support of a technical

person to be able to use this virtual museum 1 3 2 I found the various functions in this virtual mu-

seum were well integrated 4 4 4

I thought there was too much inconsistency in this

virtual museum 2 2 1

I would imagine that most people would learn to

use this virtual museum very quickly 4 5 4

I found the virtual museum very cumbersome to

use 2 1 1

I felt very confident using the virtual museum 4 4 5 I needed to learn a lot of things before I could get

going with this virtual museum 2 2 1

Table 2: The scores of the usability questions.

(17)

4.2.2 Interviews

All of the testers thought that the virtual museum was easy to use once they had figured out how to interact with objects. They generally thought the GUI and overall usability were well made. One of the testers said it was difficult to figure out which button on the Oculus Go pointer to use for the interactions. The other two testers said that the birds were difficult to notice, but once they found the birds, it was easy to understand that they were interactable.

The suggested improvements for the virtual museum were mostly about the birds. One suggestion was to add a particle effect or sound to the birds while they fly out of the painting to make them more noticeable. Another suggestion was to increase the size of the glowing effect displayed while the pointer is pointing at a bird. The only suggestion not related to the birds was to explain how the Oculus Go pointer is used.

4.2.3 Observations

For the most part, the testers could use the virtual museum without difficulties.

All of them learned how to use most of the features quickly. The painting on the opposite wall was found very quickly by all testers. Noticing the birds and discovering that they were interactable was probably the most difficult part of the experience for the testers. Two of the testers had to be told that the birds existed or could be interacted with. One of the testers did not attempt to interact with the objects and had to be told to interact with the paintings. The same tester did not notice the audio buttons at first and had a bit of difficulty pointing at the buttons with the pointer.

(18)

5 Discussion

Virtual museum experiences have been shown to make the visitors feel more im- mersed and get more enjoyment out of a museum visit. The tests performed for this museum project appears to indicate this as well since the testers would like this kind of experience to be part of a real museum. A virtual museum experience could probably be a nice addition to a real museum. The virtual environment can contain any type of object, not just paintings. This makes the virtual experience useful for more than just art museums. Any type of museum could use a virtual museum experience to show objects and stories in a more interactive and fun way for the visitors.

5.1 Usability

The usability of the virtual museum was good when the testers had either learned or been told how to use some of the features. The biggest problems were noticing the birds and learning how to use the Oculus Go pointer to interact with objects.

Because of these problems, a lot of visitors of a museum might not be able to experience everything in the virtual environment. Another problem that might affect the experience of the visitors is the audio buttons. Since one of the testers had a bit of difficulty using the buttons, other people might have the same problem.

If a visitor has a difficult time playing the audio, it could potentially ruin the experience for the visitor.

While working on a project, everything is usually easy to use and understand since it quickly becomes very familiar to you. A lot of usability problems tend to be overlooked because of this. Performing tests with users who do not know anything about the project is a very effective way of discovering some of these problems.

5.2 Improvements

A quick tutorial explaining how to use the Oculus Go pointer could be added to the project. This would probably be able to teach the users how to interact with objects using the pointer. An explanation that glowing objects are interactable could also be useful information to add to the tutorial to improve usability. A particle effect could be added to the birds while they are flying out of the painting to make the birds more noticeable. A sound could be useful to add to the flying birds as well to make it more clear that something is happening. The audio buttons could be improved by making them bigger. If the buttons were bigger, the user would probably be more likely to notice the buttons. The buttons would also be easier to interact with if they were bigger.

(19)

5.3 Social, ethical, and environmental considerations

This section discusses social, ethical, and environmental considerations about the usability and the testing of the virtual museum.

5.3.1 Usability and Improvements

All the text and narration in this virtual museum is in Swedish. If a visitor does not understand Swedish, this would be a problem. An option for selecting language could be added to the project to make the experience usable to more people.

Since the GUI for interactive objects supports both text and audio, visitors with difficulty reading would still be able to understand. The same goes for visitors with difficulty hearing, they can just read the text to get the information. If a visitor has really bad eyesight, it might not be possible for them to play the audio or even interact with objects. This virtual museum would probably not be usable in that case.

5.3.2 Testing

Everyone who participated in the testing of the virtual museum volunteered to test the experience. Each tester was informed that they were observed and that the Oculus Go would record their actions to be reviewed later. All parts of the testing were treated with anonymity.

(20)

6 Conclusion

This prototype of a virtual museum experience needs more development to be used in a real museum. There are a couple of usability problems that have to be addressed before visitors would be able to use this without any help or difficulties.

A few other usability improvements could also be made to improve the experience for the visitors.

The tests performed on this prototype show that there is interest to have a virtual reality experience as part of a museum. This would probably be a new and fun way of experiencing a museum for a lot of people. This project mainly focused on paintings, but all kinds of objects could be added to the virtual environment.

All kinds of museums could have use for this kind of experience, not only art museums.

(21)

7 References

[1] Wolfgang H¨urst, Xhi Jia Tan, and Ferdinand de Coninck. Using Digital Ex- tensions to Create New VR Museum Experiences. In Proceedings of the 13th International Conference on Advances in Computer Entertainment Technol- ogy, pages 1–6, 2016.

[2] Chelsea Kelling, Otto Kauhanen, Heli Väätäjä, Jussi Karhu, Markku Tu- runen, and Vesa Lindqvist. Implications of Audio and Narration in the User Experience Design of Virtual Reality. In Proceedings of the 22nd International Academic Mindtrek Conference, pages 258–261, 2018.

[3] MAN - Museum Anna Nordlander. http://konstmuseet.com/.

[4] Skellefte˚a museum. https://skellefteamuseum.se/.

[5] Konstn¨ar Anna Nordlander - MAN - Museum Anna Nordlander. http://

konstmuseet.com/om-man/anna-nordlander/.

[6] Unity Real-Time Development Platform | 3D, 2D VR & AR Visualizations.

https://unity.com/.

[7] Oculus Go: Standalone VR-headset | Oculus. https://www.oculus.com/

go/.

[8] Raw Image | Unity UI | 1.0.0. https://docs.unity3d.com/Packages/com.

unity.ugui@1.0/manual/script-RawImage.html.

[9] Unity - Scripting API: AudioSource. https://docs.unity3d.com/

ScriptReference/AudioSource.html.

[10] System Usability Scale (SUS) | Usability.gov. https://www.usability.gov/

how-to-and-tools/methods/system-usability-scale.html.

(22)

8 Appendix

8.1 Appendix 1 - Questionnaire

(23)

(24)

Development of an Interactive VR Experience for an Art Museum