Facebook Announced 3D Photos Feature, Uses AI To Simulate Photo’s Depth

Facebook just released a feature called 3D photos. It allows users to post images with 3D effects to their Newsfeed. The feature was introduced at F8 conference of Facebook in May 2018, but Facebook had not gone into details how the feature would work up until now. They have just released further details of the 3D photo feature that gives us a good idea about how AI (Artificial Intelligence) is utilized to simulate the depth in a typical 2D picture.

This 3D photo feature will work with photos in portrait mode and those 3D photos will be visible to both PC and mobile users and also VR with Oculus Rift or Oculus Go. You can simply click or tap on the photo and then drag to move it in order to see it in 3D.

Behind the screen

Both cameras will be used in a dual set up in portrait mode. One camera is used to capture the depth perception in order to create an effect called ‘bokeh’. However, for 3D photo feature of Facebook, AI will use parallax differences, which is ‘how an at different distances shift different amounts when the viewing angle is changed’ to simulate what is behind the object.

The designers of this system are Peter Hedman from University College London and a research scientist named Johannes Kopf Computational Photography research group of Facebook. You normally have to move your cameras if you want to capture photos like that. However, with the popular presence of dual cameras, measuring the difference in parallax between two lenses is much easier. It just put a bit more work on the CPU.

The number crunching

To capture a photo in portrait mode, each camera will both take a photo and then the phone will proceed to calculate a depth map between two taken images. The phone basically calculates the distance of every object in the photo’s frame. Smartphones of Samsung, Huawei or Apple, that feature is often used to make blur effects in the photo’s background.

The system of Facebook, users will have to move the camera around where they want to take the photo and take multiple images. The depth maps between the two for each taken image will then be analyzed by a custom algorithm and it also analyzes how much the cameras have shifted in each time. They will then be combined together and turn your 2D images you took into 3D photos that have movement and depth when users tilt their smartphone or view inside a VR headset

This feature is now only limited to users of iPhone 7 Plus, iPhone 8 Plus, iPhone X and iPhone XS but Facebook will provide this feature to a wide range of devices in the future.