🖖Hand Tracking
Hand Tracking Input
Required XR Settings for Hand Tracking
After downloading the subpackage in Auto Hand/Packages/OpenXR.package
Example will be found in Auto Hand/Examples/Scenes/OpenXR/Hand Tracking
These are the required components for OpenXR hand tracking:
XR Hand Skeleton Driver Unity's script for getting transform data from OpenXR
XR Hand Tracking Events Unity's script for getting events from OpenXR
OpenXR Auto Hand Tracking Converts Unity hand tracking data to your Auto Hand skeleton as well as smoothing for hand tracking gesture and movement
OpenXR Auto Hand Tracking Grabber Manages if grabbing / holding conditions are being met
OpenXR Auto Hand Tracking
Hand: is a reference to which hand this script is driving
Up Axis: if your skeleton on your hand rig matches the OpenXR hand tracking skeleton, this axis should equal "Up". If your skeleton is different you figure out what your hands' up axis equivalent is on the OpenXR rig. On the robot hand example the yellow up axis is equal to the OpenXR's left (opposite direction of the red arrow)
Forward Axis: same as previous value, but for your hand's blue forward arrow. In the example below the robot hand's forward axis matches the OpenXR hand's yellow up axis. So it is set to "Up"
Hand Offset: Position offset for the hand in local hand tracking space
Hand Rotation Offset: This offsets the hand relative to it's tracking target. In order for the hand tracking to work the hand model must have correct rotation offset to the tracking hand
Hand Pose Smoothing: Add a small smooth movement delay when matching the fingers to their target pose. This mainly helps the hand from moving jarringly when in using low confidence poses
Follow Position Smoothing Speed: This will smooth out the hands follow target position over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.
Follow Rotation Smoothing Speed: This will smooth out the hands follow target rotation over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.
OpenXR Auto Hand Tracking Grabbing
Touch Grab Settings
handTracker: Reference to the hand tracker responsible for tracking hand movements and poses
releaseGrabDelay: The delay in seconds before a grab is released after the grab condition is no longer met
fingerTipRadiusMultiplier: Multiplier for the radius of the finger tip detection spheres used in touch grabbing
useFingerTouchGrabbing: Enables or disables grabbing objects by touching them with fingers
useFingerTouchReleasing: Enables or disables releasing objects by touching them with fingers
useTouchHoldingWithHeldPose: Enables or disables maintaining a held pose while holding an object with finger touch grabbing
Pose Grab Settings
usePoseGrabbing: Enables or disables grabbing objects using predefined hand poses
minPoseGrabCloseness: The minimum closeness required for the hand pose to initiate a grab
maxPoseGrabCloseness: The maximum closeness allowed for the hand pose to initiate a grab
requiredDeltaPoseGrabCloseness: The required change in closeness to trigger a grab
Pose Release Settings
usePoseRelease: Enables or disables releasing objects using predefined hand poses
minPoseReleaseOpenness: The minimum openness required for the hand pose to initiate a release
maxPoseReleaseOpenness: The maximum openness allowed for the hand pose to initiate a release
requiredDeltaPoseReleaseOpenness: The required change in openness to trigger a release
Pose Squeeze Settings
usePoseSqueezing: Enables or disables squeezing objects using predefined hand poses
squeezeUnsqueezeDelay: The delay in seconds before a squeeze is unsqueezed after the squeeze condition is no longer met
squeezePoseSensitvityMultiplier: Multiplier for the sensitivity of the squeezing pose detection
Raw OpenXR Hand Tracking Input Visuals
These "Right/Left Hand Tracking Visuals" show the default OpenXR hand tracking input before it's sent to the Auto Hand Tracking components.
If hand tracking quality seems low, you can check these visuals to see if it's the hand tracking input that isn't work well or if it's the Auto Hand Tracking that isn't working as expecting
MetaXR Auto Hand Tracking
Hand: is a reference to which hand this script is driving
Up Axis: if your skeleton on your hand rig matches the OpenXR hand tracking skeleton, this axis should equal "Up". If your skeleton is different you figure out what your hands' up axis equivalent is on the OpenXR rig. On the robot hand example the yellow up axis is equal to the OpenXR's left (opposite direction of the red arrow)
Forward Axis: same as previous value, but for your hand's blue forward arrow. In the example below the robot hand's forward axis matches the OpenXR hand's yellow up axis. So it is set to "Up"
Hand Offset: Position offset for the hand in local hand tracking space
Hand Rotation Offset: This offsets the hand relative to it's tracking target. In order for the hand tracking to work the hand model must have correct rotation offset to the tracking hand
Hand Pose Smoothing: Add a small smooth movement delay when matching the fingers to their target pose. This mainly helps the hand from moving jarringly when in using low confidence poses
Follow Position Smoothing Speed: This will smooth out the hands follow target position over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.
Follow Rotation Smoothing Speed: This will smooth out the hands follow target rotation over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.
MetaXR Auto Hand Tracking Grabbing
Touch Grab Settings
handTracker: Reference to the hand tracker responsible for tracking hand movements and poses
releaseGrabDelay: The delay in seconds before a grab is released after the grab condition is no longer met
fingerTipRadiusMultiplier: Multiplier for the radius of the finger tip detection spheres used in touch grabbing
useFingerTouchGrabbing: Enables or disables grabbing objects by touching them with fingers
useFingerTouchReleasing: Enables or disables releasing objects by touching them with fingers
useTouchHoldingWithHeldPose: Enables or disables maintaining a held pose while holding an object with finger touch grabbing
Pose Grab Settings
usePoseGrabbing: Enables or disables grabbing objects using predefined hand poses
minPoseGrabCloseness: The minimum closeness required for the hand pose to initiate a grab
maxPoseGrabCloseness: The maximum closeness allowed for the hand pose to initiate a grab
requiredDeltaPoseGrabCloseness: The required change in closeness to trigger a grab
Pose Release Settings
usePoseRelease: Enables or disables releasing objects using predefined hand poses
minPoseReleaseOpenness: The minimum openness required for the hand pose to initiate a release
maxPoseReleaseOpenness: The maximum openness allowed for the hand pose to initiate a release
requiredDeltaPoseReleaseOpenness: The required change in openness to trigger a release
Pose Squeeze Settings
usePoseSqueezing: Enables or disables squeezing objects using predefined hand poses
squeezeUnsqueezeDelay: The delay in seconds before a squeeze is unsqueezed after the squeeze condition is no longer met
squeezePoseSensitvityMultiplier: Multiplier for the sensitivity of the squeezing pose detection
If you want to disable the visual hands the OVR includes disable these components found here
This component should go onto your hand along with the OVRHand and OVRSkeleton
Grab Fingers Required will trigger grab when the included fingers are bent past the given amount, and release when any fingers are less than this value
Thumb, Index, Middle, Ring, Pinky finger components are found under the hand
Free Fingers will allow fingers to move while holding an open (as long as they don't break the grab finger requirements). For example lift the index finger up while holding a soda bottle
Grab Action will call the grab event when all the included fingers are bent past the given amount. Release event is called when grab action is no longer valid.
In this case grab will occur when the index middle and thumb are bent past halfway or 0.5
Squeeze Action will trigger the squeeze event when all the included fingers are bent past the given amount.
Custom Hand Tracking Event Component
Hand Tracking includes an event script which will call Press/Release
Press is triggered if all the fingers in Finger Bend Past are bent past the bend value, and all the values in Finger Bend Below are below the bend value.
Release is triggered if press was, but is no longer valid
Controller Swapping
The OVR Hand Controller Swapper will allow you to automatically swap between controllers and hand tracking. Connect the hand follow targets for hand tracking / controller tracking then connect any objects/scripts that should be disabled on swap
Last updated