/ Dmytro Lapshyn

Mastering AR: App for Sony SmartEyeglass

Firsthand experience of software development for Sony SmartEyeglass augmented reality glasses, impressions on the use of the device, and specifics of app development for it.

There have been a lot of interest to augmented reality, which spurs the demand for this technology. Many companies have been trying their powers in AR development for several years to build up skills and competence. No one wants to jump in a moving train (i.e. well-established and mature technology).

One of the obstacles in the technology development is immaturity of AR devices available on the market. For the most part, AR devices are at an early stage of development or underworked and cost too much. We’ve decided to try the most promising devices and see whether they are ready to market. In this article, we’ll discuss Sony SmartEyeglasses.

 

Sony SmartEyeglass First Impression

The first device we got our hands on was Sony SmartEyeglass. It is designed to superimpose flat computer graphics onto the wearer’s natural field of view (something like Terminator’s vision in the first and second part of the movie). It doesn’t cover all AR purposes, but it fits such tasks as heads-up display, remote guidance, and geolocation (“the broken cell tower is 300 meter to the south-east”, “check fuses in the power supply unit” with a layout view of their location, etc.

There are many videos showing capabilities of these glasses.

Here’s one of them: http://youtu.be/F8pjS3lM0R4?t=15s

The reality is not as shiny as those videos: the device for sale now is a developer edition available for business customers only and costs $800. Sony glasses work only with mobile devices equipped with Android 4.4 or higher. All operations shown on the glasses are processed by your phone and transmitted via Bluetooth or Wi-Fi. Also, Sony has its SDK for app development, which is necessary for the glasses to see the app installed to the device. Sony website provides a clear and plain explanation of that, so let’s skip the details.

 

The App We Developed

Initial app idea was to read barcodes with glasses camera and get full product info: name, country of origin, price, weight, etc. First, experimenting with such Terminator sight was fun, but with time we understood that barcode reading itself in AR context has little practical use. So, our next version was much more useful for daily life. The app scans product codes and adds them to a virtual cart to see the total cost of the purchase before a check out.

 

A great help in development was Sony’s comprehensive documentation of glasses setting and development of software for them. Developer’s guide also includes some sections specific for augmented reality. The trick is to show information on glasses display without distracting a user. For example, a person driving a car or piloting a plane cannot shift attention to augmented reality information for a long time. App developers need to consider it and keep in mind what a user is going to do when using the app: walking, sitting, or something else. For every activity type, Sony provides guidelines: if a person is moving, the app should display as little text as possible and in a large font. Further, a user needs to understand, what app level they are at. That’s why Sony recommends to avoid deep nesting.

We need to note that Sony imposes many constraints, so we couldn’t work directly with the camera to receive real time stream and show a marker to determine barcode edges on the screen. So, to recognize barcodes we took a photo by the camera and then searched for the barcode in the database. This method is of course not so expedite, but it was the only one to solve the task at that moment.

UI limitations have also made an impact. The elements stocked by Sony couldn’t meet all of our requirements, so we used Sony dialog box as a menu for our app. It worked out pretty well.

 

Results

It was an interesting experience both for us, developers, and for designers, who worked on UI for the app, because that was our debut in AR. The device is still in its pupilage and provides little opportunities for super innovative solutions. However, it’s truly amazing to work in this area even now.

In general, our vision of the market niche for these glasses is industrial application and, probably, remote guidance for DIY projects.


More firsthand experience in development for Microsoft Hololens and Epson Moverio to come in the next few weeks. Don’t miss!

Dmytro
Lapshyn
Subscribe for regular updates