DeepAR SDK methods

Detailed docs here.

DeepAR(settings)

Used to initialize the DeepAR SDK. Example usage:

var deepAR = DeepAR({
    canvasWidth: window.innerWidth,
    canvasHeight: window.innerHeight,
    canvas: document.getElementById('deepar-canvas'),
    numberOfFaces: 1,
libPath: './lib',
segmentationInfoZip: 'segmentation.zip',
    onInitialize: function() {
      // called when the initialization is finished
    }
});

// SegmentationInfoZip is relative to the lib path.

Settings:

  • canvasWidth, canvasHeight: width and height of the canvas surface where the camera feed with AR objects will be rendered

  • canvas: canvas DOM object

  • onInitialize: the function which will be called when the initialization is finished. No other API should be used before that

  • numberOfFaces: how many faces the SDK will simultaneously track. The maximum is 4 and the more faces it tracks, the worse performance becomes.

  • libPath: relative path to the lib folder. The folder contents must be the same as the lib folder from the SDK download

  • segmentationInfoZip: there is a segmentation.zip file inside SDK lib folder. One must provide this path if you are going to use background segmentation features of WebSDK

downloadFaceTrackingModel (path, callback)

Downloads the face tracking model and initializes the face tracking. When it's done, the callback is invoked.

loadSegmentationModel(callback)

Preload the background segmentation model if needed.

switchEffect (face, slot, path, callback)

Downloads the effect file and loads it into the engine. The method used to switch any effect in the scene.

The face the parameter is used to specify the on which face the effect will be applied. The allowed values are 0,1,2,3 - 0 is the first face detected, 2 the second, etc.

The slot specifies a namespace for the effect in the scene. In each slot, there can be only one effect. If you load another effect in the same slot the previous one will be removed and replaced with a new effect. Example of loading 2 effects in the same scene:


switchEffect(0, "mask", "https://my_site.com/flowers", {})
switchEffect(0, "glasses", "https://my_site.com/glasses", {})


Using this method users can combine multiple effects in the scene. When the effect is downloaded and loaded into the engine, the callback will be called.

clearEffect(slot)

Clears the given slot of any loaded effects.

processImage(image)

If you want to apply DeepAR processing on a single image instead of a camera stream use this method. The image parameter is the HTML IMG dom element. See example usage on our Github pages.

startVideo (mirror, videoOptions)

Starts the video feed. The video feed will be horizontally mirrored if mirror=true. Parameter videoOption will be passed to getUserMedia, like this:

navigator.mediaDevices.getUserMedia({video: videoOptions})

Here's the order of events which will be called after startVideo:

  1. onCameraPermissionAsked

  2. onCameraPermissionGranted or onCameraPermissionDenied

  3. onVideoStarted

setVideoElement (videoElement, mirror)

Used to pass the HTMLVideoElement to the DeepAR SDK. The SDK will grab frames from the video element and render the frames with masks/filters to the canvas. This method should be used instead of startVideo when you want to handle getUserMedia outside of the SDK or you need to apply the masks to any video stream.

stopVideo ()

Stops the video feed.

pause()

Pauses the processing loop.

resume()

Resumes the previously paused processing loop.

shutdown()

Shuts down the engine. Make sure to call if you do not want DeepAR processing frames anymore.

fireTrigger (name)

This method allows the user to fire a custom animation trigger for model animations from code. To fire a custom trigger, the trigger string must match the custom trigger set in the Studio when creating the effect. 

changeParameter (gameObject, component, parameter, value)

This method allows the user to change parameters during runtime. The method requires a Game Object name, Component name, Parameter name and a float value.

  • gameObject is the name of the Game Object in the hierarchy as visible in the Studio. The name should be unique for the method to work properly.

  • component is the internal component type. Currently, the only supported component type for this method is "MeshRenderer". The component must exist on the game object.

  • parameter is the name or the parameter to be changed, for example, the name of the blendshape on a mesh. The parameter must exist for the component.

  • value is the new value to be applied to the parameter.

// Change the weight of the blendShape1.cuteBS blendshape on the Mesh Game Object to 5.0 (5%).
deepAR.changeParameter('Mesh', 'MeshRenderer', 'blendShape1.cuteBS', 5.0);

changeParameterVector (gameObject, component, parameter, x, y, z, w)

Same as above except this method can be used to change the value of the four-component vector. Can be used to change material color. If multiple Game Objects share the same material, changing the parameter once for any Game Object using the material will change it for all Game Objects.
To change a uniform on a material, such as color, the parameter must use the internal uniform name. These names can be found in the shader files.

// Change the color of the u_diffuseColor uniform (diffuse color for Standard shader) on the material applied to the Mesh Game Object to a semi-transparent green.        
deepAR.changeParameterVector('Mesh', 'MeshRenderer', 'u_diffuseColor', 0.0, 1.0, 0.0, 0.25);

It's also possible to change the transform of the object. In that case, the component parameter should be an empty string. Here are some examples:

// Change the position of the Mesh object       
deepAR.changeParameterVector('Mesh', '', 'position', 0.1, 0.0, 0.0, 0.0);
// Change the rotation of the Mesh object      
deepAR.changeParameterVector('Mesh', '', 'rotation', 45, 0.0, 0.0, 0.0);
// Change the scale of the Mesh object      
deepAR.changeParameterVector('Mesh', '', 'scale', 2.0, 2.0, 2.0, 0.0);

changeParameterBool(gameObject, component, parameter, value)

This method allows the user to change parameter at runtime of type Boolean. The most common usage is to enable/disable certain node/gameObject in the effect node hierarchy. Example:

deepAR.changeParameterBool('base_lips', '', 'enabled', false);

changeParameterTexture(gameObject, component, parameter, textureUrl)

This method allows the user to change the image parameter for a game object at runtime. The image parameter must be a valid image URL. Example:

deepAR.changeParameterTexture('lips', 'MeshRenderer', 's_texColor', 'https://my_website.com/pink_lips.png');

The parameter name is the name of the texture uniform found in the shader used on this game object.

moveGameObject(selectedGameObject, targetGameObject)

Moves the selected game object from its current position in a tree and sets it as a direct child of a target game object.

setFaceDetectionSensitivity (sensitivity)

This method allows the user to change face detection sensitivity. The sensitivity parameter can range from 0 to 3, where 0 is the fastest but might not recognize smaller (further away) faces, and 3 is the slowest but will find smaller faces. By default, this parameter is set to 1.

takeScreenshot ()

Captures a screenshot of the current screen. Event onScreenshotTaken will be called after it's done.

setCanvasSize (width, height)

Changes the canvas size. It should be called after the engine is initialized (onInitialize has been called).

Events

onInitialize()

The method is called when the engine is initialized.

onScreenshotTaken(photo)

Called when the screenshot is taken. The photo parameter is the result of HTMLCanvasElement.toDataURL("image/png").

onCameraPermissionAsked()

Called when the camera permission is asked.

onCameraPermissionGranted()

Called when the camera permission is granted.

onCameraPermissionDenied()

Called when the camera permission is denied

onFaceVisibilityChanged(visible)

Called whenever the face enters or exits the camera field of view.

onImageVisibilityChanged ()

Called whenever the image enters or exits the camera field of view.

onVideoStarted ()

Called when the video is started.

onFaceTracked (faceData)

Called for each camera frame with the face data.

onNumberOfFacesVisibleChanged (numberOfFaces)

Called when the number of detected faces changes

onError (errorType, message)

Called when any DeepAR related event happens.

Did this answer your question?