The concept came to me, when I first started to experiment with the Rokoko Suit. I wanted to make a game, which takes the advantage of VR as well as the real time motion caption technology of the Rokoko Suit. From the beginning I was sure what kind of a technology I would develop for this task. I wanted a completely out of body experience, where the player would control the body and see through the eyes of a digital character, as their own. Soon after iterating different ideas, I came up with the idea of recreating a TV show game, I saw as a kid, called Hole in the Wall. The concept of the game was simple, where a wall approaches the player, with a silhouette cut out, through which the player must go through. This proved to be a perfect fit for the kind of tech I was planning to develop using the real time motion tracking and VR technologies. The game would make the player move their limbs and head, to go through the different silhouette cutouts, which takes full advantage of the technology we had. The disadvantage of our mo-cap suit was, that it does not handle root motion (physical movement of the player in the real world) very well. Thus this gameplay also ensures that the player themselves do not move too much, and the core mechanic would be moving your limbs, and not yourself as a whole.

The first step is to connect the Rokoko Suit to your computer, via the same WiFi. Once connected, you can see the suit transmitting the movements live into Rokoko Studio (the desktop app by Rokoko). 

Next, we got Rokoko Studio to transmit live into Unity, where we have a rigged character. We reproject the movements from the Rokoko suit, being captured live to our character in Unity. 

The character is made in a way, where each limb has a collider, and the camera is fixed to the head bone of the character. Using this we create gameplay in Unity, where the walls approach the player in a random order (we had 15 unique walls in total). If the player goes through the silhouette cutout in the wall without touching the wall, their score increases. If the player does touch the wall, the wall breaks, and is now segmented randomly into small chunks. Over here, the player can interact with these chunks, which have real time physics implemented.

Lastly,  we project our Unity gameplay live onto the Oculus RIft, through an HDMI cable. The fact that the camera is placed in the head of the character, the player always sees from the first person perspective of the digital character/avatar.

FLOW CHART: Process and pipeline