As part of a course, I developed a MR (mixed-reality) museum experience that turns artwork interpretation into a spatial interaction.
Role: Designer / Developer (Individual project)
Course: Emergent User Interfaces
Platform: Meta Quest 3
Tools: Unity, Meta XR SDK, MRUK
Duration: 4 weeks
Balancing guided learning with visitor autonomy
The attention challenge
Traditional museum information displayed on labels and plaques often require visitors to shift their visual attention away from the artwork. With MR, I wanted to explore if placing interpretation directly within the artwork can create a more cohesive experience between looking and learning.
How research shaped my design principles
Desktop research and my ethnography at Helsinki's Ateneum art museum shaped the backbone of the interactions.
1. Visitors' engagement varies a lot — between-subject
During an extended observation at Ateneum, I noticed that engagement with wall texts varied widely throughout different visitors. This suggested that guidance should be optional so that it accommodates different levels of interest.
2. Visitors' engagement varies a lot — within-subject
During the observation, I noticed that even within the same person, engagement with different artworks can vary a lot. This suggested that information should be self-directed and bite-sized.
3. MR can break the spatial limitations of interpretation
There are existing samples of special MR exhibitions that leveraged spatial annotations and overlays to provide immersive experiences.
1. Visitors' engagement varies a lot — between-subject
During an extended observation at Ateneum, I noticed that engagement with wall texts varied widely throughout different visitors. This suggested that guidance should be optional so that it accommodates different levels of interest.
2. Visitors' engagement varies a lot — within-subject
During the observation, I noticed that even within the same person, engagement with different artworks can vary a lot. This suggested that information should be self-directed and bite-sized.
3. MR can break the spatial limitations of interpretation
There are existing samples of special MR exhibitions that leveraged spatial annotations and overlays to provide immersive experiences.
I grounded my design principle as the following:
Autonomy-preserving,
Guide instead of interrupt,
Spatially relevant.
Autonomy-preserving,
Guide instead of interrupt,
Spatially relevant.
Hesper, the Artsy Ghost
With these principles in mind, I started designing a playful MR guide that reveals information as visitors choose to explore. The core interaction is triggered by proximity as the user approaches the artwork, then interaction points will be visible so the user can pick which area to learn about.
The interaction point activates a combination of visual aid and audio guide to inform the viewer of the selected area. Throughout the interaction, there is a silence button available to the user to mute all aids to simply observe.
It is also possible to compare the artwork with another work placed side-by-side. This works really well in showing a related work. With this example, Hesper allows visitors to compare how Picasso created his rendition of Las Meninas.
User testing and iterations
User testing boiled down to two main iterative changes that I included in the final prototype.
1. Users couldn't always tell which area an interaction point referred to. The spatial relationship between the point and its corresponding area was not as apparent as I assumed. To address this, I added outlines that would show on hover to make the relationship explicit.
2. Users wanted to interact with the dog in Las Meninas. In the painting, there is a dog that is napping in the corner. A couple users tried to click on the dog, and I realized that not everything interesting needs to be educational. So, I added a snoring soundtrack to an interaction point beside the dog, as a little "easter egg".
1. Users couldn't always tell which area an interaction point referred to. The spatial relationship between the point and its corresponding area was not as apparent as I assumed. To address this, I added outlines that would show on hover to make the relationship explicit.
2. Users wanted to interact with the dog in Las Meninas. In the painting, there is a dog that is napping in the corner. A couple users tried to click on the dog, and I realized that not everything interesting needs to be educational. So, I added a snoring soundtrack to an interaction point beside the dog, as a little "easter egg".
Demo
Reflections
The prototype left me curious about what makes people interested enough to seek out information voluntarily. If I were to continue on this project, I would extend the experience to sculptures and allow visitors to move around an artwork and discover different layers of information from different angles.
Ultimately, I explored MR as a tool for making learning about art feel more natural and intuitive. While MR museum guides are not yet practical for actual adoption due to their scarcity, this project served as a personal experiment of whether I can design a seamless experience with a peripheral technology that might feel "extra".
Ultimately, I explored MR as a tool for making learning about art feel more natural and intuitive. While MR museum guides are not yet practical for actual adoption due to their scarcity, this project served as a personal experiment of whether I can design a seamless experience with a peripheral technology that might feel "extra".