Game log 4

Brief documentation of and what you prepared for the Expo

We developed an augmented reality game, where players fish out fragmented parts of a painting from a virtual lake and piece them together to reconstruct the artwork. The objective of the game is to collect and assemble parts of a painting that are scattered within a virtual lake. Players must use the AR interface to 'fish' for painting fragments and then solve the puzzle by arranging the pieces in the correct order.

Players interact with the game through VR-headset and controllers, which overlays the virtual lake onto a real-world environment. By aiming with the right controller players can fish out parts of the painting. Once all parts are collected, players enter a puzzle-solving mode where they must arrange the fragments to complete the painting with their left hand. Upon successful completion of the puzzle, a brief description of the painting and its cultural or historical significance is displayed.

Exhibition Setup

To enhance the player experience and create a more immersive and tangible interaction, we constructed a physical representation of the lake using paper. Before we start the game we can move the lake with the left controller to a position where it fits the best to the real world environment.

User Feedback and Engagement

To further assess the enjoyment and engagement level of our game, we also prepared a survey for participants to fill out after playing. The survey aimed to gather feedback on gameplay experience, the educational value of the painting descriptions, and overall enjoyment.

Brief Technical Description of the Playable Prototype

Our playable prototype is built in Unity, supporting both controller input and hand tracking. The experience combines fishing mechanics with 3D puzzle assembly. 

Challenges & Solutions

1. Combining Hand Tracking and Controller Input Simultaneously

One challenge was enabling simultaneous use of a controller in the right hand and hand tracking in the left hand. This is not the default behavior in OpenXR, as they typically expect either two controllers or two tracked hands. We needed to then carefully select the correct settings, so that the hand tracking does not interfere with controller tracking. It is now possible to grab with the right hand, however, you are not supposed to do that, because the fishing rod will attach to your hand once the right controller is put away and it looks weird. That is why the placing of the puzzles and if the puzzle is correct is only handed in the left hand.

2. Grabbable Object

Since we were using the Grabbable Script from InteractionSDK and some deprecated Building Blocks, some things did not work as planned. After the shrinking of an object, the grabbable object behaves in a weird way. The box collider will not be the same and grabbing objects with a box collider that is not equal to the visual box of the object is not very satisfying. In the end, we did not have to shrink anymore, because we made everything smaller.

3. Making the Fishing Rod Look Realistic

It was hard to set up everything correctly to make the fishing rod behave realistically. When extending the rope that connects the hook to the rod, we initially tried to move the hook along the axis between the tip of the fishing rod and the hook’s current position. However, the hook’s gravity settings were not configured correctly, which caused it to drift in unintended directions (sometimes left, right, or even upward).

We also explored whether to move the hook using the local or world coordinate system. Using local coordinates caused problems when the rod was rotated, as the hook would then extend in the wrong direction. World coordinates used consistent downward movement, but didn’t allow the hook to move forward in the direction of the rod.In the end, we decided on a solution where the hook is "thrown" by positioning it directly along the same line as the rod (as if the rope were just an extension of the rod itself) and then letting gravity take over from there.

4. Positioning the lake with the left controller

Before we start the game (or during the game) we have the ability to position our lake the place where it fits the best to the real world environment. To achieve this, we needed to track the position of the left controller and move the lake to that position which was not a challenge, but previously we enabled hand tracking and controller tracking at the same time, which caused the first trouble.

In our first solution the lake followed the left controller continuously so we could see where we move our lake in “real-time”, but after we put down the controller it goes to sleeping mode, so the lake suddenly jumped to the player’s left hand and followed every movement of it.

To overcome this problem, we managed to implement a solution where the lake doesn’t follow the controller continuously but jumps to it’s position when we press the ‘X’ button on it and the lake stays there until we press the button again at a different location. Through this implementation we learned a lot about hand, controller, position tracking and button mappings.

The next problem we had to face was this odd behavior of the lake after we placed it. It seemed that it follows the user wherever he moves. The solution for this problem was not too technical but required some research to figure out that it was caused because the lake y position was lower than 0, so we had to introduce some offset to solve this problem. 

Evaluation of the game

We created a questionnaire for the players who tested our game at the EXPO. Each of them filled it in right after playing. In total, our game was evaluated by 13 respondents.

Part 1: Background information

We asked each respondent for their age, experience with VR, and experience in game development to gather background information about the group. The players had a very diverse experience with VR, while three players never used it before, five used it occasionally, and five have used it regularly. 

Experience with computer games development also varied. Most participants (eight) had developed games as part of their education, while three had done so as a hobby. Two respondents had professional experience in game development and two reported no experience at all.

This diversity makes the feedback valuable not only for understanding how accessible and intuitive the experience is to a broad audience, but also for gaining insightful advice from experts. 

(One of the players skipped this question.)

Part 2: Game experience

All 13 respondents found the game somehow enjoyable.

What do you think about the combination of fishing and puzzle elements?

The idea was mostly received positively. Respondents described it as original, creative, fun, engaging, and effective. One of the respondents said: “It actually works quite well. It kind of has a great framework for the game mechanics. Because you easily understand what to do and what the concept is. Story wisely could maybe be better integrated why fishing - but it works.” 

How did you feel about using a controller in one hand and hand tracking in the other?

This was a multiple-choice question with the option to write your own answer. Five players said it worked well, while two found it confusing. We also received several individual comments: “it was a bit unresponsive”, “odd to pinch and rotate at the same time with the secondary hand (left)”, “it is a little weird but it works alright since it is a fishing tool.” 

One respondent noted that it worked well but felt unfamiliar because they are left-handed. This is something we hadn’t considered while designing the game. It could be improved by allowing users to select their dominant hand at the beginning.

What aspects of the experience did you enjoy the most?

Four people mentioned fishing, three enjoyed putting the painting together, and one mentioned both activities. 

What would you improve or change in the experience?

We received several suggestions for improvement. A number of participants recommended adding more pieces of art, sound feedback or haptic responses to increase immersion. 

Some players addressed the need of improving snapping of the puzzle elements. Collected paintings often overlapped, making it difficult to see if a new piece was fished successfully or not. One respondent recommended improving the storytelling by for example showing that we are a sailor who had been paid to deliver paintings but the ship is sinking and we want to save them. Participants also noticed some bugs in both fishing and puzzles. 

Part 3: Questions about the painting

Can you describe the final painting in your own words?

The respondents described the painting correctly. However, most of the descriptions were not very detailed. One player even wrote that they had forgotten what exactly the painting looked like because they were focused on the task.

What facts do you remember about the painting from the final text?

Five respondents remembered a single fact about the painting (often that it was made in 1918, or that there is a metaphor between fishing and freedom). Four did not remember any facts and four players did not answer the question at all. This shows that, if the game is used for educational purposes, more immersion and interaction with the text would be needed.