🖖Hand Tracking

Hand Tracking Input

OpenXR (Recommended)

Required XR Settings for Hand Tracking

After downloading the subpackage in Auto Hand/Packages/OpenXR.package

Example will be found in Auto Hand/Examples/Scenes/OpenXR/Hand Tracking

These are the required components for OpenXR hand tracking:

XR Hand Skeleton Driver Unity's script for getting transform data from OpenXR

XR Hand Tracking Events Unity's script for getting events from OpenXR

OpenXR Auto Hand Tracking Converts Unity hand tracking data to your Auto Hand skeleton as well as smoothing for hand tracking gesture and movement

OpenXR Auto Hand Tracking Grabber Manages if grabbing / holding conditions are being met

OpenXR Auto Hand Tracking

Hand: is a reference to which hand this script is driving

Up Axis: if your skeleton on your hand rig matches the OpenXR hand tracking skeleton, this axis should equal "Up". If your skeleton is different you figure out what your hands' up axis equivalent is on the OpenXR rig. On the robot hand example the yellow up axis is equal to the OpenXR's left (opposite direction of the red arrow)

Forward Axis: same as previous value, but for your hand's blue forward arrow. In the example below the robot hand's forward axis matches the OpenXR hand's yellow up axis. So it is set to "Up"

This example shows why the "Up/Forward" Axis settings are "Left/Up" on the robot hand tracking

Hand Offset: Position offset for the hand in local hand tracking space

Hand Rotation Offset: This offsets the hand relative to it's tracking target. In order for the hand tracking to work the hand model must have correct rotation offset to the tracking hand

Hand Pose Smoothing: Add a small smooth movement delay when matching the fingers to their target pose. This mainly helps the hand from moving jarringly when in using low confidence poses

Follow Position Smoothing Speed: This will smooth out the hands follow target position over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.

Follow Rotation Smoothing Speed: This will smooth out the hands follow target rotation over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.

OpenXR Auto Hand Tracking Grabbing

Touch Grab Settings

handTracker: Reference to the hand tracker responsible for tracking hand movements and poses

releaseGrabDelay: The delay in seconds before a grab is released after the grab condition is no longer met

fingerTipRadiusMultiplier: Multiplier for the radius of the finger tip detection spheres used in touch grabbing

useFingerTouchGrabbing: Enables or disables grabbing objects by touching them with fingers

useFingerTouchReleasing: Enables or disables releasing objects by touching them with fingers

useTouchHoldingWithHeldPose: Enables or disables maintaining a held pose while holding an object with finger touch grabbing

Pose Grab Settings

usePoseGrabbing: Enables or disables grabbing objects using predefined hand poses

minPoseGrabCloseness: The minimum closeness required for the hand pose to initiate a grab

maxPoseGrabCloseness: The maximum closeness allowed for the hand pose to initiate a grab

requiredDeltaPoseGrabCloseness: The required change in closeness to trigger a grab

Pose Release Settings

usePoseRelease: Enables or disables releasing objects using predefined hand poses

minPoseReleaseOpenness: The minimum openness required for the hand pose to initiate a release

maxPoseReleaseOpenness: The maximum openness allowed for the hand pose to initiate a release

requiredDeltaPoseReleaseOpenness: The required change in openness to trigger a release

Pose Squeeze Settings

usePoseSqueezing: Enables or disables squeezing objects using predefined hand poses

squeezeUnsqueezeDelay: The delay in seconds before a squeeze is unsqueezed after the squeeze condition is no longer met

squeezePoseSensitvityMultiplier: Multiplier for the sensitivity of the squeezing pose detection

Raw OpenXR Hand Tracking Input Visuals

These "Right/Left Hand Tracking Visuals" show the default OpenXR hand tracking input before it's sent to the Auto Hand Tracking components.

If hand tracking quality seems low, you can check these visuals to see if it's the hand tracking input that isn't work well or if it's the Auto Hand Tracking that isn't working as expecting

Disable these components to hide the ghost hands

MetaXR

MetaXR Auto Hand Tracking

Hand: is a reference to which hand this script is driving

Up Axis: if your skeleton on your hand rig matches the OpenXR hand tracking skeleton, this axis should equal "Up". If your skeleton is different you figure out what your hands' up axis equivalent is on the OpenXR rig. On the robot hand example the yellow up axis is equal to the OpenXR's left (opposite direction of the red arrow)

Forward Axis: same as previous value, but for your hand's blue forward arrow. In the example below the robot hand's forward axis matches the OpenXR hand's yellow up axis. So it is set to "Up"

This example shows why the "Up/Forward" Axis settings are "Left/Up" on the robot hand tracking

Hand Offset: Position offset for the hand in local hand tracking space

Hand Rotation Offset: This offsets the hand relative to it's tracking target. In order for the hand tracking to work the hand model must have correct rotation offset to the tracking hand

Hand Pose Smoothing: Add a small smooth movement delay when matching the fingers to their target pose. This mainly helps the hand from moving jarringly when in using low confidence poses

Follow Position Smoothing Speed: This will smooth out the hands follow target position over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.

Follow Rotation Smoothing Speed: This will smooth out the hands follow target rotation over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.

MetaXR Auto Hand Tracking Grabbing

Touch Grab Settings

handTracker: Reference to the hand tracker responsible for tracking hand movements and poses

releaseGrabDelay: The delay in seconds before a grab is released after the grab condition is no longer met

fingerTipRadiusMultiplier: Multiplier for the radius of the finger tip detection spheres used in touch grabbing

useFingerTouchGrabbing: Enables or disables grabbing objects by touching them with fingers

useFingerTouchReleasing: Enables or disables releasing objects by touching them with fingers

useTouchHoldingWithHeldPose: Enables or disables maintaining a held pose while holding an object with finger touch grabbing

Pose Grab Settings

usePoseGrabbing: Enables or disables grabbing objects using predefined hand poses

minPoseGrabCloseness: The minimum closeness required for the hand pose to initiate a grab

maxPoseGrabCloseness: The maximum closeness allowed for the hand pose to initiate a grab

requiredDeltaPoseGrabCloseness: The required change in closeness to trigger a grab

Pose Release Settings

usePoseRelease: Enables or disables releasing objects using predefined hand poses

minPoseReleaseOpenness: The minimum openness required for the hand pose to initiate a release

maxPoseReleaseOpenness: The maximum openness allowed for the hand pose to initiate a release

requiredDeltaPoseReleaseOpenness: The required change in openness to trigger a release

Pose Squeeze Settings

usePoseSqueezing: Enables or disables squeezing objects using predefined hand poses

squeezeUnsqueezeDelay: The delay in seconds before a squeeze is unsqueezed after the squeeze condition is no longer met

squeezePoseSensitvityMultiplier: Multiplier for the sensitivity of the squeezing pose detection

If you want to disable the visual hands the OVR includes disable these components found here

SteamVR

IndexHandControllerLink

Grab Fingers Required will trigger grab when the included fingers are bent past the given amount, and release when any fingers are less than this value or when the grab action is triggered/untriggered

Oculus Integration (Deprecated)

The hand tracking Demo can be found within the Oculus Integration package. It will be found in Auto Hand/Examples/Scenes/Oculus Integration/Hand Tracking

If you can't find that scene follow the setup guide for Oculus Integration

This component should go onto your hand along with the OVRHand and OVRSkeleton

Grab Fingers Required will trigger grab when the included fingers are bent past the given amount, and release when any fingers are less than this value

Thumb, Index, Middle, Ring, Pinky finger components are found under the hand

Free Fingers will allow fingers to move while holding an open (as long as they don't break the grab finger requirements). For example lift the index finger up while holding a soda bottle

Grab Action will call the grab event when all the included fingers are bent past the given amount. Release event is called when grab action is no longer valid.

• In this case grab will occur when the index middle and thumb are bent past halfway or 0.5

Squeeze Action will trigger the squeeze event when all the included fingers are bent past the given amount.

Custom Hand Tracking Event Component

Hand Tracking includes an event script which will call Press/Release

• Press is triggered if all the fingers in Finger Bend Past are bent past the bend value, and all the values in Finger Bend Below are below the bend value.

• Release is triggered if press was, but is no longer valid

Controller Swapping

The OVR Hand Controller Swapper will allow you to automatically swap between controllers and hand tracking. Connect the hand follow targets for hand tracking / controller tracking then connect any objects/scripts that should be disabled on swap