Augmented reality (AR) software tips: How to get a better AR experience

The quick answer is to **smoothly shift your device horizontally left and right as you move **.

** Explanation**
If you’ve tried an AR app before and never gotten it to behave properly, understanding how ARKit works could help you get better results.

ARKit streams HD video at 60 frames per second from the camera while using the motion sensors at a higher rate to detect the direction and amount of movement between frames and finally using the 3D graphics processor to create 3D objects that it merges into the scene and updates at 60 frames per second. In short, it’s all wonderful magic.

To place an object in the video stream at the correct size and location, ARKit needs to understand the 3D structure of the scene in the video stream. It does this by comparing two different frames and using left-right parallax to measure the depth of points in the scene that it is tracking. This works best when the two frames are both sharp and differ only in their horizontal position. Fast or jerky motion creates blurry frames. Likewise, out of focus frames and the noisy images from low light do not work well. ARKit really benefits from brightly lit scenes.

Note that shifting your device left and right is different from rotating it left and right. Rotation of your device around the camera center does not cause the parallax shift important to ARKit.

ARKit requires some seconds to acquire sufficient information to properly track a scene. During this start up period, smoothly moving your device horizontally either left or right around the scene can provide a better AR experience.

Much of the information in this post comes from my year of AR app development and from the excellent 2018 WWDC session 610: [Understanding ARKit Tracking and Detection](https://developer.apple.com/videos/play/wwdc2018/610/)

1 Like

Hi George. Thanks for this. I am just starting to explore this field.
I would like to make AR instructional videos of sewing procedures I do in my business.

Can you help me? Where do I start? Is this even possible? This is a totally new dimension for me :slight_smile:

I have an iPhone 6S.
Any efforts you can give me to point me in the right direction would be appreciated!

Maripat

Hi Maripat,

Sorry for taking so long to respond. I missed seeing your message. :disappointed:

I’m not sure at all from your description what you have in mind. Do you want to be able to point the phone’s camera at different parts of a sewing machine and have a sign popup telling what that part does and how to use it? If so, that’s certainly possible with AR, but I’m not really a good one to help you with this as I don’t have experience with object recognition.

All the best,
George.

More companies are becoming interested in AR. Nintendo is a great example. They were one of the pioneers of AR in the mobile space when Pokémon Go launched in 2016.