Hooked on History

Hooked on History is an augmented reality game where players fish fragmented parts of a painting from a virtual lake and piece them together to reconstruct the full artwork. This playful combination of fishing and puzzle-solving turns passive viewing into an active and embodied interaction with art.

Players use a VR headset, a controller in one hand, and hand tracking in the other. With the right controller, they cast and reel in hidden painting fragments from the lake. Once all pieces are collected, the player switches to puzzle mode, using their tracked hand to arrange and rotate the fragments until the painting is complete. Upon successful assembly, the game presents historical context about the artwork to have a moment for reflection after interaction.

---

The Cultural Heritage Concept – What’s the story?

Our goal was to let players interact meaningfully with art, beyond just observing. By making them assemble the painting themselves, we encourage them to pay attention to details, textures, and symbolism they might overlook in a museum setting.

The chosen painting touches on themes of freedom:

• Freedom of expression – art resisting censorship
• Political freedom – tied to social or democratic movements
• Freedom of identity – celebrating diversity and self-expression

These themes are not just textual — they emerge from the painting itself, and through the act of “rescuing” it from the lake, players symbolically engage with its preservation and meaning.

---

The AR Prototype – Interaction techniques and technical challenges

The game runs in Unity using the Oculus Interaction SDK with OpenXR, combining controller-based fishing and hand-tracked puzzle solving.

Main technical challenge: enabling simultaneous use of a controller in the right hand and hand tracking in the left. This is not default behavior in OpenXR, so we had to manually configure input sources to ensure they didn’t interfere.

Other implementation challenges included:

• Making the fishing rod behave realistically while controlling the hook
• Anchoring the virtual lake in the physical world (not just to the controller)

---

The Game Experience – How close did we get?

We tested the prototype with 13 players at an expo. Their feedback shows we’re on the right track:

• All 13 players found the game enjoyable
• Most described the fishing + puzzle combo as creative, intuitive, and engaging
• Input method (controller + hand tracking) was praised but unfamiliar to some

Suggested improvements:

• More feedback (e.g. sound, haptics)
• Clearer snapping for puzzle pieces
• Better narrative context
• Left-handed player support

Educational impact: Most players could describe the final painting, but retained only a few facts from the explanatory text — suggesting we should strengthen the final storytelling moment.

---

Cover Image generated with ChatGPT 4o