VR/AR – 5 Important Cases to Consider
VR and AR are confidently taking their place among other human-machine interfaces that we use on a daily basis. As of today, we are no longer speculating whether VR/AR applications are worth creating, we would say that we are already past that point. It’s already here, these technologies are moving from Early Adopters to Mass Market stage, and we expect the wave of VR/AR development to emerge.
However, being a software engineering company and providing consulting services to our clients, we have singled out several challenges related to these technologies. So let’s take a quick look at obstacles that VR/AR still need to overcome to become a full-fledged mass market technology.
1. Expensive hardware
You don’t have to be a prophet to predict that hardware price decrease will play a major role in VR technology spreading. VR poses significant challenges for GPU manufacturers. While fast and beautiful graphics are important for any truly immersive experience, this is especially so for VR, where slower response time may cause VR motion sickness and ultimately render the product impossible to use.
Let’s take a look at some numbers. According to PCgamer.com “most people perceive a flickering light source as steady illumination at a rate of 50 to 60 times a second, or hertz”. To make the same natural effect in motion computer graphics, this figure was transformed to FPS (frames per second) rate, the required bare minimum for VR applications is 60 FPS. However, “…out in the periphery of our eyes we detect motion incredibly well. That’s partly why VR headsets, which can operate in the peripheral vision, update so fast (90 Hz).” (PCGamer.com “How many frames per second can the human eye really see?”)
Atman Binstock, Chief Architect at Oculus, explains further: “Traditionally, PC 3D graphics has had soft real-time requirements, where maintaining 30-60 FPS has been adequate. VR turns graphics into more of a hard real-time problem, as each missed frame is visible. Continuously missing framerate is a jarring, uncomfortable experience. As a result, GPU headroom becomes critical in absorbing unexpected system or content performance potholes.”
No wonder that with these requirements we have to deal with the cutting-edge hardware, causing early adopters to pay a fortune for a VR-ready PC (according to Logical Increments: No Compromises Build – the most powerful consumer PC available, suitable for VR games, streaming, 4K video editing and 3D modelling and animation will cost $5,300).
2. Outside-in and Inside-out tracking
In Outside-in tracking (Oculus Rift, HTC Vive, PS VR), the sensors are placed in different corners of the room to provide full coverage and determine the position of the headset in relation to the environment. In real life, it means that you have to allocate a room, which will be equipped with the necessary set of cameras, a PC station and a headset to allow you to immerse into virtual reality. What is the probability rate that this setup can be used in a regular household? Very low, indeed. Therefore, the predictable volume of applications development for the headsets with Outside-in tracking will hardly reach the mass market.
Inside-out tracking offers more flexibility for a user, since a Head Mounted Display (HMD) already possesses a built-in camera which looks out to determine the position in relation to the external environment. Consequently, it drastically changes the situation with the cameras setup as in the case with Outside-in tracking. You simply don’t need them anymore.
One of our favorite devices with inside-out tracking is HoloLens. It’s a Mixed Reality device which has 1 depth camera as well as 4 environment understanding cameras allowing environment rendering with more sophistication. A see-through option makes it possible to see the surrounding environment, a little bit blurred though, but nevertheless the user can benefit from the immersiveness of the untethered holographic device and at the same time overcome the uneasiness of being isolated as in the case of a VR headset.
Here is an interesting story that we had with HoloLens. Sigma has developed Interio application which is aimed at helping designers, architects, and event organizers perform interior visualizations and create an overall look-and-feel of the future premises. We have successfully showcased the solution at VR Sci Fest which took place at the Royal Institute of Technology in Stockholm in May. But unfortunately, after one day of the exhibition, HoloLens crashed and could not be restored. Let’s face it – the device lifetime and maintenance is a vital component. Although we still deal with beta version and development kit, not a final retail product. But again, you have to be sure that the final product is a stable and reliable device, proving its value for the price.
3. Optimal resources utilization
The battery life is a stumbling block for lots of the devices. With mobile VR, which is more affordable for a great number of people, the issue with power consumption must be resolved with the highest priority.
According to CNet.com “The electronics that enable faster performance, higher-speed data, better video and gaming, a more vivid and detailed screen, are moving at the speed of Moore's Law. The lithium ion (Li-ion) pouch cell batteries that power them can't keep up. Little wonder that battery life is the biggest complaint of smartphone users!”
Taking this into account as well as the required FPS rate which should be set at 60 minimum (usually it requires 30 FPS for mobile games or even less), it’s quite easy to realize that mobile VR will just devour the battery charge in a couple of hours. This challenge can be partially overcome by means of efficient application code, smart and consistent app design, and graphics assets. The other part of the problem should be solved with the hardware on a global scale.
4. User experience
VR and AR are actually a new human-machine interaction paradigm. Just think about it – when you are working with an application in a headset, you have to re-consider a standard user interface, e.g. buttons selection is different from what we are used to. We don’t really do the clicking any more. Depending on the device you either gaze at the object or use motion controls or perform voice commands. To close the window you can, for example, do a throw-away gesture and to open file you grab it by the file name.
And think about the display itself. Most of us are used to working with quite a small display size – say, around 15 inches for laptops on average. While Virtual Reality is the display around you. The files or windows (will that be still ‘windows’?) can open all around you, or just one window in front of your eyes, as wide as you are comfortable working with its content.
By gradually shifting to new VR/AR interfaces, we will surely adopt new UI/UX interaction paradigms.
5. Relevant use cases
With all the hype traditionally accompanying VR and AR, let’s stop for a moment and think about the applications that proved to be inevitable for everyday life. Like, for example, we can’t really go on without mobile phones today. You feel lost and in silo if your battery died and there’s no way to reach your contacts or use your favorite apps.
When we were preparing this article, our team of Unity developers obtained a possibility to test ARKit which will be available in iOS 11 version. The experience was amazing! From our perspective, we will have a surge of AR development as soon as the ARKit is out. As we already see it today, the ARKit greatly simplifies the development work required for creating AR apps by offering superb surfaces recognition, not to mention the wide iPhone user base where this SDK will be available.
Now, the most difficult goes to creating compelling use cases. Will you really be using AR apps to check out some objects on surfaces? The technology behind it is really unrivalled. Apple will allow iPhone users experience AR firsthand, fast and easy. The next biggest challenge is to create use cases that will make users cling to that apps, transform them into everyday life helpers or attributes.
Summing it all up, technologic challenges will inevitably be solved with the progress, which is really fast-paced now. And the challenge of discovering valuable use cases, applications or products that you won’t be able to imagine your life without is what we all need to face now. Our aim in this respect is a strong collaboration with an enthusiastic product owner. The product management and development perspectives play the crucial role. At Sigma Software we strive to take ownership in product development and work in tight cooperation with the product owner to create apps that will last.
See description of AR apps we developed:
INTERIOR VISUALIZATION APP BASED ON MICROSOFT HOLOLENS
SHOPPING ASSISTANT APP BASED ON SONY SMARTEYEGLASS
This article is a joint effort of Iegor Antipov, an AR/VR expert, and Natalya Zheltukhina, as a word painter and technology enthusiast.