Hand Tracking

Hand tracking enables users' hand gestures as the input for YVR headsets, thereby striking more natural and engaging interactions between apps and users, so as to enhance the immersive VR experience. You can integrate the hand tracking feature into your app, so users can click, select, drag and drop, and perform other operations in your apps with simple hand gestures such as poke, pinch, and hold.

Note

Hand tracking is only supported for 64-bit applications. Before using this feature, under Edit > Project Settings > Platform > Android > Build, make sure to check Support arm64 and uncheck Support armv7.

Setup Steps

  1. In the top left of the editor, click Edit > Project Settings > Plugins > YvrXR Settings.

  2. Under the Controller section, set HandTracking Support to Controllers and Hands / Hands Only.
    InputType

Add Hand Tracking Componenets

You can add the YvrXRHand component to a Pawn or Actor as needed.

  1. Double click on the Pawn or Actor you would like to add the YvrXRHand component.

  2. In the top left corner of the screen, click + Add Component, enter "XRHand" in the search box, and then add it to the component directory of the Actor or Pawn.
    YvrXRHand

Set Hand Tracking Components

Once you have added the component for the Pawn or Actor, you will need to select the Pawn or Actor and make the following settings in its Details panel.

  1. Under the Mesh section, click the drop-down button to the right of Skeletal Mesh and select the skeletal mesh you need to assign to the corresponding component.
    YvrHandMesh

  2. Under the Hand Properties section, click the drop-down button to the right of Skeleton Type and select a skeleton type. Check Hide by Confidence and Update Hand Scale.

  3. If you are using BP_XRHandComponent_Left and BP_XRHandComponent_Right, under the Custom Skeletal Mesh section, hand tracking nodes have already bound the hand component's bone key.
    YvrBoneMapping

Sample

The Hand Tracking sample project sets you up to create a simple hand tracking demo. It illustrates how to use the YVR hand tracking feature.

Please follow the steps to view the sample:

  1. Open the VRTemplate.uproject.

  2. Connect to YVR device.

  3. Select Launch to start the sample project.