A look into ARCore

A group of mobile devices with digital objects placed in their camera views using augmented reality

Our interactive development team at NYU IT has been very active in our explorations of mixed reality since January, 2018. This past week we decided to explore ARCore to better understand the capabilities and limitations of current augmented reality technology. We’ve been testing it using Unity 3D and a Pixel 2, and here’s what we found:

Overall, ARCore has a bunch of nice features. Object tracking is very well done. Augmented objects are securely fixed to the plane they are placed on. Even from angles where the ground plane is out of view, the augmented objects can still be seen allowing you to get a closer look at the underside of objects. Additionally, multiple surfaces can be tracked at once, which gave us the idea to explore objects jumping from one surface to another. ARCore also has the ability to map objects to uneven surfaces, in addition to floors, walls, and ceilings. Finally, ARCore brings objects to life by applying a dynamic lighting that tries to match the color profile of the camera. That way, if you view augmented objects in a dimly lit room or a room with colored lights, the objects will appear to be colored to match the real-life environment around it.

While ARCore is very powerful, there are inevitably some drawbacks that come with it. If an object is mapped to a surface and the surface begins to move, the object will not move with it. Furthermore, the illusion of depth is lost when something passes between the camera and the object because the object gets drawn on top of the obstacle, even if the obstacle should be closer than the augmented object in real life. Lastly, ARCore is currently only for Android devices. To do AR on iOS, you should instead explore Apple’s ARKit. As we move forward in our explorations, we’ll continue to look for methods to push the platform and extend what we’ve discovered to be some of best enhancements augmented reality has to offer; the feeling of intimacy and scale of normally inaccessible things (animal predators, ancient architecture, etc.), placed in the same space as the user.