Image tracking

DeepAR supports image tracking. To track an image the user needs to have a copy of the image printed on a surface or displayed on a screen, the image can then be detected and 3D objects can be rendered at the position of the image in real time

The image should also be:

  • non-rotationally-symmetric

  • rich in detail

  • non-repetitive

  • on a flat mat surface

Image tracking can be used concurrently with face tracking, which means it can be used to create complex effects with independent objects.

To create an effect with image tracking the RootNode must be positioned in Camera space, and the 3D model whose position will be driven by the image position, or it's parent node must have the Natural Image Position component attached. The image to track can be set in the Component parameters under the Image Classifier parameter.

To track multiple images create multiple gameobjects, one corresponding to each image and add the Natural image component with the desired image to each of them.

Natural Image Position component setup

Image tracking example tutorial

In this tutorial, we will demonstrate how to build image tracking effect with DeepAR studio.

At the end of this section, there is an example folder with DeepAR studio file that contains all assets mentioned here. You can download it and use it to try this effect or create your own image tracking effect. 

1. Import FBX file 

  • Open the DeepAR Studio v1.7.4. or above

  • Now import FBX via File → Import FBX

  • Add the FBX file from the example folder (Image_tracking_example/fireworks.fbx)

2. Texture animation

Now, load the texture animation 

  • Go to the textures tab Add →Animation

You can choose any animation file you want. In this example, we will use Image_tracking_example/animation

  • Choose Image_tracking_example/animation folder and click Open and Done

3. Setup Node properties

We don't need the face tracking for this effect so select the RootNode and in the node properties set Position mode to the position in camera space.

4. Add image for tracking

  • Select Root Node and go to the Components tab

  • Select Natural Image Position and click Add

  • Now, select the Natural Image Component in the list below, select the Image Classifier property and click Select file

  • You can choose any image file. In this example, we will use Image_tracking_example/starbucks.jpg

  • After clicking Open, it will take some time for the engine to process the image.

5. Setup animation effect 

  • Go to the material tab and choose lambert42  

  • Under Select shader drop-down lists choose "Unlit texture"

The unlit texture is there to only render texture without any lighting.

  • For Diffuse texture choose animation which we added before

  • Next, select pPlane1 node under the Node hierarchy 

  • Go to the Components tab and add Play Animation Continuous component

Play Animation Continuous means that the animation plays in an endless loop

  • Again, select pPlane1 node and set it's scale to 15 (x,y and z-axis) to make it bigger

6. Final results

If the chosen image ( in this example starbucks.jpg ) is in the camera field of view, you will see the fireworks animation.

To immediately see the image tracking effect, load Image_tracking_example/image_tracking file in the Studio by clicking File, Open effect.

Download this example with texture assets

Examples with face tracking

In this example, we did a little fun with image tracking by adding one of the DeepAR products from the Assets store. For effect to show up we choose Jokester face filter and for tracking image joker card. This is a funny way to summon Jokester from the joker card. Take a look at the video below.

Summon cute creature via the DeepAR logo? Why not, it's never been so easy.

Did this answer your question?