vr & biosignals: increasing interpersonal understanding via a vr co-op driving game
Team: Hyun Woo Paik, Wei Gong
(October 2018 - December 2018 [Development: December - December])
independent study, ux research & design, game design, unity, C# programming
Visual Studio (C#)
Fraps (for recording)
iMovie (for concept video)
My team sought to achieve interpersonal understanding via a collaborative experience in a VR game. We rapidly ideated, conceptualized, and developed the initial version of a timed 2-player co-op VR driving game where two players must match heart rates to achieve a common goal. We did this while learning Unity for the first time.
It was overall a challenging effort that could have benefited from developing prototypes earlier on, but I learned a lot about development versus conceptual design, and when to do one vs. the other.
In a virtual reality environment, people often lack the cues to express their emotions or to feel or know the emotions of others. For many, a voiceover conversation does not always give clues into a user’s true emotional state. My team sought to answer the following question:
How can we increase the emotional and social interpersonal understanding between two players via Biosignals (such as heart-rate) in a VR environment?
conceptual answer in unity (v0.5)
We created the initial version of a co-op driving game where audio and visuals are changed based on the player heartrate. Two players must both lower their heartrate, ideally by calming each other down, in order to reach the finish line in time, all while avoiding obstacles.
The driver cannot see the objects in the road, while the passenger can. If the driver hits an object, the car slows down.
When the driver’s heart-rate increases, the music in the car also increases, making it more difficult to communicate. Through this, the passenger gets an idea of what the driver is currently experiencing.
exploring emotions in games & social context
Our initial work involved exploring emotions in games, from a third-person and first-person perspective. This started with Psychologist James Russell's circumplex model of affect, which describes distribution of human emotions in a two-dimensional circular space. In this model, emotional states can be represented by any level of valence and arousal, or at a neutral level of one or both of these factors.
The task of combining emotions into both narrative and gameplay was a challenging one, especially considering the constraints of VR.
As a team, we took emotional concepts and built scenes together, with each one of us adding an element to a particular scene. This helped us come up with different ideas collaboratively. Our first few storyboards focused more on narrative and stimuli: specifically, what prompted characters to behave in certain ways.
ideating via storyboarding: phase 1
Our initial work involved exploring emotions in games, from a third-person and first-person perspective. The task of combining narrative and gameplay was a challenging one, especially considering the constraints of VR. We each created our own storyboards and shared our ideas, building on top of each other's ideas.
ideating via storyboarding: phase 2
In order to finalize a concept, we started with many storyboards (16+, a sample of which is shown above [illustrated by Hyun Paik and Wei Gong]) and narrowed it down to one concept: a driver and passenger in a car who have to work together by syncing their biosignals (in this case, heart-rate BPM) to successfully reach the finish line within a time limit
Once we narrowed down our concept, we created further storyboards for speed-dating purposes (depicted storyboards by Hyun Paik, me, and Wei Gong, respectively). Speed-dating was done to identify user needs when it comes to interpersonal understanding. We experienced a lot of difficulty in designing a game that also addresses emotional needs in a VR context.
However, we did gain 2 valuable insights:
Diminishing the vision of drivers will force them to listen to passengers. Whenever there is an argument between a driver and a passenger, the driver usually has the upper hand because they have control. Majority of the participants agreed on an idea of reducing the vision of a driver as a means of “forced” communication with a passenger.
Showing just heartbeat by number is unclear and easy to miss. Heartbeat is a great indicator of the physical state of a person, but participants were unclear of what heartbeat numbers meant and the range of each emotional state.
prototyping & game design
When we were designing the rules for the game and prototyping scenarios, we tried to answer the following questions:
What are the prompts for the two players to communicate with each other?
How would they communicate and interact with each other? Are there any obstacles for the communication?
How can biosignal synchrony help with their communication and interpersonal understanding?
I was in charge of audio and displaying information to the player. I added music and sound effect files to game in Unity using C#. I set up music to start/stop with key command (if not playing at start of game) and controls to manually adjust volume (as the first version was more of a Wizard of Oz concept). Set up sound effects and mic input, with ability to change mic volume and pitch (which broke/was buggy in Unity). I also up BPM count that mimicked integration with Biosignals (went up when the player hit an obstacle, and could also be manually adjusted.)
Controls (version 0.5):
Left Arrow: Decrease mic pitch
Right arrow: Increase mic pitch
Up arrow: Increase radio volume
Down arrow: Decrease radio volume
Spacebar: Turn radio On/Off
Q: Decrease BPM
W: Increase BPM
I: Driver POV
O: Passenger POV
P: High BPM skyfilter
There are two players in the game, a passenger and a driver. They load into the same car, while the car is already moving. The two players need to reach the terminal point of the road covered with random obstacles as fast as possible to win the game together. Each time the car hits an obstacle, the speed of the car will slow down.
Therefore, the goal of the driver is to control the car to avoid the obstacles to ensure that the car is in high speed. However, the driver cannot see the obstacles whereas the passenger can see them. Thus, the goal of the passenger is to guide the driver to turn left and right to avoid the obstacles. They need to better communicate with each other in order to achieve the goals.
The volume of the music in the car changes along with the heart-rate of the driver. When the heart-rate of the driver is higher, the volume of the music is higher, impeding their communication with each other. Therefore, in order to hear the guidance of the passenger and control the car appropriately, the driver needs to try to calm themselves down when the music is louder. In addition to an onscreen BPM indicator, the color of the sky will change when the driver’s heart-rate reaches >110 BPM, which is another reminder for the driver to calm down.
reflection & future direction
Our project was very well-received, which was a relief considering all the challenges we encountered, such as:
It was our first time learning and using Unity.
Unity for Mac and Windows are not compatible. We ended up manually re-entering all code.
We spent a lot of time debugging and fixing the driver display issues before we could continue with the project, as the computer did not meet minimum spec requirements to develop for VR.
For future versions:
Implement multiplayer control. The different views of the two players are currently switched manually (with keypress).
Implement heart-rate tracking bracelet. The heart-rate is controlled manually (with keypress).
Clarify that the BPM on the screen indicates the heart-rate of the driver. Additionally, might need to use a better indicator than “BPM” as this may not be common knowledge.
Considering the short timespan we were given to learn how to develop, create and import assets and audio in Unity, etc., I believe we ended up with a new, fresh concept to further iterate upon.