Camera to be the world’s first, mainstream AR platform
Approximate Reading Time: 4 minutes
The 32-year old CEO of Facebook, Mark Zuckerberg, today unveiled Facebook’s plans for transforming the way we interact with the world around us. These plans and more were unveiled at Facebook F8, an annual developer conference that brings together developers from the world over.
Zuckerberg explained that in his view, augmented reality (AR) was the future, and that Facebook’s goal would be to bring this future to life.
Speaking of the potential of AR, Zuckerberg described instances where you could, say, transform your house into the Hogwarts school of witchcraft and wizardry, or view a 50-inch TV in your living room without actually owning one and more.
Zuckerberg added that he believed, as did most people, that AR would first arrive via some form of glasses (think Google Glass). However, he notes that AR has arrived, that it’s in the hands of everyone and, surprisingly, doesn’t involve glasses. AR, says Zuckerberg, is on our phones. More specifically, it’s enabled by our phone’s camera, and thus, this will be the focus for Facebook in the years to come.
He describes our current AR experience as primitive. Primitive not out of choice, but because we’re in the process of learning and building better tools, tools that don’t exist yet.
Zuckerberg wants to build these tools and give them to developers today.
The heart of this AR transformation will be the camera.
In fact, Zuckerberg went so far as to say, “If there’s one thing you’re going to take away from the event, it’s that the camera will be the first mainstream AR platform.”
Zuckerberg describes three primary use cases for a camera. The first is simply information. This includes messages, directions, etc. The second is adding digital objects to our real world. The third involves the enhancement of existing objects.
Even with the technology available today, achieving these goals isn’t easy. Barring a rare few exceptions, the camera of today still see in 2D. This makes positioning anything in AR rather difficult. To make matters worse, adding information, enhancing a scene or even adding a digital object requires a deeper understanding of the scene.
To all of this, AI comes to the rescue.
Using a technique called Simultaneous Localisation and Mapping (Slam) and combining it with cognitive neural networks, Facebook has created AI that can process a scene in 3D in real-time, even if it’s only got a photo to go on.
The technology itself is extremely fascinating, but we’d rather not delve into it right now. Suffice it to say that the new algorithms are far more precise than anything we’ve ever experienced.
As advanced as all of this is, the results might at first seem to be rather lame. Doesn’t it seem a shame to employ the genius of some of the greatest researchers and scientists in the world just to add digital smoke to a cup of coffee and share that image on Facebook?
Honestly, it does, but these seemingly petty use cases underscore the work that researchers have done to make this a reality. The simple fact that an AI algorithm can do this is achievement in itself, but the fact that this AI can now be so streamlined as to run off the processing power of a smartphone is something else entirely.
All of these tools will be available to developers as part of the Camera Effects Platform, which will include Frames Studio and AR Studio, both of which will help developers build the new generation of Facebook experiences, and all of them in AR.
The potential for this technology is immense. As Zuckerberg demoed on-stage, imagine you’re in a waiting room with your kid. Facebook’s AR tech can transform an empty table into a virtual playground for you to play games in. The same technology can be used to dramatically enhance games, create new forms of art, interaction with friends and family and so much more.
And so far, we’re only scratching the surface.
AR is the future, but AI is its backbone, and Facebook developers will have all the resources they need to let their imaginations run wild.