Lab weeks - Zoetermeer
For the big final of this course we have spent two weeks in Zoetermeer where we were working in SCRUM system in order to create innovative games and experiences with usage of such technologies like Virtual Reality, Augmented Reality, Arduino and coding!
In overall we have been working in teams separated in themes: AR/VR, Artificial Creatures, Wearables, Exortion Gaming. I was working in a team of 5 on Augmented Reality and Virtual Reality experiences. We were working full 10 days from 9:00-17:00, mastering our skills in tools like Unity and Oculus and creating our product which we presented during evening of 10th day. We had our ups and downs during this project, figuring out how everything works but at the end we never gave up! Below I would like to go through this time day by day.
Day 1
Unfortunately I needed to do really important legal stuff on first day of our Zoetermeer lab weeks. To be cool I informed teachers about this fact and did my best during rest of lab weeks.
My group set up everything needed to start working on bigger project like this. For communication we used WhatsApp. Moreover we created our Gmail especially for this project in order to keep our files on drive.
After first brainstorming group updated me on WhatsApp about theme for the game. They chosen Kill Bill's scene when (spoiler) Beatrix was trying to kill O-Ren Ishii, but she needed to kill her ninjas first. Our idea was very similar at the beginning. We wanted to combine this scene with game Fruit Ninja. The goal would be to kill ninja's in VR like in Kill Bill movie. Game was supposed to be Kill Bill's Experience.
Moreover they tested Oculus to see how it works and what are the possibilities of this technology. At the end to see what is on the market we did research on similar games to our initial concept.
Day 2
On day 2 I finally mede it to Zoetermeer where I could meet my new team members from Dutch CMD. At the beginning of each day we were setting up Oculus with PC. However on this day it was my first time when I tried VR for longer time. After game with WALL-E like robot I was amazed with the experience this product gives and unlimited possibilities.
Later we get shared vision on idea about the game. We decided to go really into Kill Bill's vibes. I researched more on music from this movie and how it can be uploaded to Unity.
Just after all of us played VR for some time we started figuring out how we, ourselves can create VR environments like this. It turned out that with Unity it's very simple and can even be done without coding.
After few hours and couple tutorials we could create VR game where plater could walk using Oculus joysticks and explore the terrain, see his hands in VR and how they move, grab and throw objects using Earth like physics. Check out our first tries.
Felly testing first prototypes. On this stage she can see cubes on place of her hands, and she can rotate them using her hands.
In this movie you can see that I have human-like hands inside VR instead of cubes and I can move them like in real life. Moreover we can grab object and throw them.
While on day one I was stressed that I have really basic knowledge about VR, Unity and coding in general, and that it will be hard to create anything, on this day I got very excited as creating VR worlds turned out to be easier that I thought. With such mindset we finished this day.
Tutorials and learning resources:
-Roll a Ball 1
Day 3
We started day 3 with bad news. Chris told us that our game does not bring anything new and is simply another game. We were really depressed at the beginning as all of us were excited about Kill Bill theme, but we can understand Chris point of view.
We did not give up but started with brainstorming session in order to create new ideas for game. At the beginning we did not had any innovative ideas but with time, we were talking more and figure out that we can connect Virtual Reality with Augmented Reality.
At the end of the day we were clear on the fact that we are doing game that involves more than one player and connects AR with VR. In initial concept idea was that one player is in Virtual Reality and is trying to escape some room. In order to do that he needs to solve the quizes in game. Only person who can help him solve them is another player, which use his smartphone and AR. On this stage the concept was really unconcrete but we had clear direction in mind.
In general we spent day 3 brainstorming, researching and talking. Such day was crucial for our project.
Day 4
On day four we came with motivation to work on new concept. We divided group in two teams where girls were working on VR environment and quizes for the game. Male part of team was responsible for AR and technical part of the concept.
The biggest question for this day was: "How we can stream AR screen (smartphone) to VR game. For me even Unity was still pretty new software that I did not feel comfortable with and we were already doing advanced stuff. We started this day with tutorial about using phone camera as input. You can check tutorial below:
It turned out that working with Unity and smartphone is way harder that we thought and with Maurits we were stoping believing in doing technical part of the concept.
Thankfully on this day we received a huge help of older UX student Bram. He had really good skills in Unity and coding. I spent many hours allowing Maurits Samsung to work with Unity. We needed to download different SDK's and play with Unity set-up.
With Bram we were working almost whole day on this. We talked a lot with him about our concept and he suggested to make one Unity project for AR game, and another Unity project for VR game and figure out the details later. His opinion was that creating one Unity project that includes using Samsung as AR tool, which screen is being displayed inside VR environment might be hard to create technically and with our limited skills and time.
At the end of the day I was thinking if we can actually create such complicated flow with AR and VR. We finished on downloading Vuforia plugin for Unity which allows to create AR experiences using Unity in seconds.
Day 5
On day five we made big progress. At the beginning of the day we went through Vuforia tutorials and created our first Augmented Reality with Unity. I was really impressed how smooth this works.
After some problems we were able to instal app on Maurits Samsung which was adding specific 3D object above pattern we uploaded in Vuforia Portal using built-in camera. For testing we used zebra-like pattern and displayed 3D robot above it. We all got really excited when we saw how this works and that it is actually not that hard.
On this day our female part of the team was also testing VR environment with Oculus. They did great job with creating basic space and adjusting light. We decided to use scary/creepy theme for the game. In Unity we only needed to play with height of the player and size of the room, but besides that, testing part went smooth.
At the end of the day I was trying with Maurits to add microphone to the game. Because communication between VR player and AR player was problematic since beginning we decided to make it as simple as we can. In the worst case scenario we decided that players can communicate physically, without using microphones etc.
On the other hand, from research we saw that players who are listening or talking with people outside of VR, are being less immersed in VR environment. We wanted to make them communicate with microphone and earphones. That's why we decided to try adding microhpne to VR Unity game, but instead of giving microphone to VR player we would be giving it to AR player, VR player would only hear AR player but could not talk to him. AR player will be able to talk to VR player but he will not have possibility to hear him. It also makes sense in terms of plot of the game, as VR player is trapped in other dimension and he can receive a signal but not send it.
We were using Unity's Microphone function. We had some problems with it but it was some Unity versions problem and at the end we knew that it is possible to connect microphone like this.
At the end of the day we knew how to create AR environments and that we can connect microphone and hear our own voice in unity. We also tested first VR worlds and got really motivated for next week.
Later in the evening Maurits sent a video that we can create AR clues that are only visible when looking from right perspective. Such idea is really great for our concept as it makes AR player more involved. Check the video below where you can see "6" only if you are looking from specific side.
Day 6
We started next week of Zoermeer Lab Weeks full of motivation. After weekend full of thinking about our game we decided to change some things in the game. First of all we decided to use Discord for communication for purposes of expo. It was the simples way to make communication between AR player anf VR player, and because we had limited time we decided to use simple solutions. When starting a game we were simply making Discord call from computer connected to Oculus to AR smartphone. We muted the microphone for VR player so only he can hear AR player, but VR player cannot talk to AR player.
Me myself started looking in the Internet how we can create levers for our game. However I did not find any free resources and we did not that good how to create detailed 3D designs. On the other hand I found this cheap and perfect for our project levers from asset store.
Moreover during this day we discussed with group about quizzes we will implement in our VR game. We decided to do only 2 instead of 3 as time was going fast and we were still learning about Unity and VR/AR,
As you can see on picture we decided to do one quiz with levers, where user need to do right sequence and AR person will only receive numbers (ex. 2,4,5). AR player will tell VR player these numbers, and VR player will need to figure out that there are 5 levers so he will probably need to do some sequence with it.
Second quiz was about color sequence. Again AR will see some color blocks and will need to tell it to VR player. He will need to change color cubes so it match with AR hint.
As our game is kind of an escape room we decided to include a timer. When the time would be 00:00 player would see End of Game screen. We used nice bloody font to make it look real.
Video of first AR hint already designed to fit our game's theme.
At the end of the day we decided that tomorrow we need to have at least one VR game ready so we can show something on the expo.
Day 7
We started next day with quick sketches about vision of our game and we made sure that everyone has the same vision.
We bought the levers puzzle game with script from asset store, as mentioned previously. Although this asset was made for normal PC video game we decided to adjust it for VR environment.
That is what I was doing almost whole day with Maurits. W had a lot of struggle to make levers grabble and make everything work. We needed to change inputs to VR and adjust other controls. At the end of the day we could grab the levers, test and reset the quiz.
Other part of group had also hard day. Felicia was making tutorial on using multiple scenes in Unity in order to play and hide menu and end game screen when needed. Maud was working also in Unity on creating game with cubes and Tevin was creating designs for AR game.
Video of our second AR hint
Video of Tim testing our levers quiz
Day 8
On this day we had special visitors in our working space - kids! As we predicted VR was the most interesting technology for our young friends. We lost a couple of hours on entertaining them but it was fun anyway!
But that's not all we did on this day. Maud recreated her project into our VR project so we had basis for second quiz with colors and with Maurits we were working on changing the script so it works with VR. We also made plan for next day and decided on how are we going to present on expo.
Day 9
At the beginning of this day we uploaded AR hints on patterns designed by Tevin.
This day was day of final touches. From progress we implemented menu screen, added lanterns so when quiz is solved a lantern will light up. We added scary sounds in the background and zombies!
Moreover me and Maurits had a lot of small coding stuff. We made sure that the doors will open only when two quizes are solved we made sure that both quizes work properly, so levers and blocks with colors. Also we did a lot of testing and were adjusting the code so there is no bugs in the game. We also learned how to export the game so we do not need to play it from Unity but as a normal application.
Day 10 - Expo
Finally the expo day! In the morning we decided that we should have more patterns for AR game to make it more interesting for him. We created more of them and uploaded some zombies above them. We distributed patterns around area. I also created a special script that made zombie run on VR player when he opened a door. This was our 'wow' effect and lots of players got really scared. One of my friend Gilermi got so scared that he was sitting on the ground for 5 minutes.
Video of Tim playing VR game with Joel on AR during Expo.
My short description of product
Game Help Me is a two player experience including one person playing in AR (Augmented Reality) and one in VR (Virtual Reality). Person playing in VR is trapped in other dimension, by putting Oculus glasses on, he finds himself in small room full of moving zombies, doll's head and hospital bed. By playing longer he will discover that he can interact with some elements like levers or blocks in wall.
Person in AR is in our dimension and can communicate (voice) to person in VR. The communication is only one way, meaning that VR player can only hear but cannot talk. AR player needs to find patterns distributed in real space, scan them with smartphone and pass what he sees to VR player. He can see things like numbers or color sequences, but also dead zombies, grrr!
VR player hearing different sequences of numbers and colors needs to think how he can use them to solve some quizes in order to escape this creepy room. By trying and failing he eventually can figure out quizes only to find out that there is no escape!
Game is filled with scary sounds, animated zombies and unexpected experiences.
Team video
Reflection
I am really amazed what we achieved. Before Lab Weeks I had really basic knowledge about Unity and no knwoledge about using Unity in VR experiences. At the end of these 2 weeks I was able to move freely in Unity, write Scripts and simply to create amazing VR environments.
I believe our product is truly innovative. It involves two players while one is in AR and other in VR. I did not see that much of such experiences in the Internet. I also like the metaphor we used: VR player is in another dimension (because being in VR actually feels like it) and AR player is in our dimension.
When somebody asked me what is the goal of our game, answer was simple: "to scary people so they cannot sleep :)". During expo we could hear lots of screams, people sitting on the ground being shocked, we heard guys screaming like girls and people ending game because of being too scared. I guess this is the answer to question: "did our game achieved its goal?".
I really enjoyed these lab weeks. It was hard work but I do not remember time when I learned so much in such a short period of time. Moreover working with group was just great. Everyone found what he/she feels good at and was putting effort to achieve shared goal. For example I felt really good in coding and designing for VR in Unity. I believe that I can simply understand new coding languages and I used this skill. While other part of group did great job on how everything looked, me and Maurits did a great job on how everything worked.
Besides that I enjoyed working in open space environment. Sometimes when we were out of motivation, checking other teams and talking with them was helping us a lot.
The expo day was simply amazing. Looking how others people play your game and interact with environment that we designed for them is a feeling I did not experienced before.