# Hand Tracking
Hand Tracking Input
{% tabs %}
{% tab title="OpenXR (Recommended)" %}
{% hint style="info" %}
**Required XR Settings for Hand Tracking**\
After downloading the subpackage in **Auto Hand/Packages/OpenXR.package**
Example will be found in **Auto Hand/Examples/Scenes/OpenXR/Hand Tracking**
{% endhint %}
These are the required components for OpenXR hand tracking:
**`XR Hand Skeleton Driver`** Unity's script for getting transform data from OpenXR
**`XR Hand Tracking Events`** Unity's script for getting events from OpenXR
**`OpenXR Auto Hand Tracking`** Converts Unity hand tracking data to your Auto Hand skeleton as well as smoothing for hand tracking gesture and movement
**`OpenXR Auto Hand Tracking Grabber`** Manages if grabbing / holding conditions are being met
## OpenXR Auto Hand Tracking
**`Hand`**: is a reference to which hand this script is driving
**`Up Axis`**: if your skeleton on your hand rig matches the OpenXR hand tracking skeleton, this axis should equal "Up". If your skeleton is different you figure out what your hands' up axis equivalent is on the OpenXR rig. On the robot hand example the yellow up axis is equal to the OpenXR's left (opposite direction of the red arrow)
**`Forward Axis`**: same as previous value, but for your hand's blue forward arrow. In the example below the robot hand's forward axis matches the OpenXR hand's yellow up axis. So it is set to "Up"
This example shows why the "Up/Forward" Axis settings are "Left/Up" on the robot hand tracking
**`Hand Offset`**: Position offset for the hand in local hand tracking space
**`Hand Rotation Offset`**: This offsets the hand relative to it's tracking target. In order for the hand tracking to work the hand model must have correct rotation offset to the tracking hand
**`Hand Pose Smoothing`**: Add a small smooth movement delay when matching the fingers to their target pose. This mainly helps the hand from moving jarringly when in using low confidence poses
**`Follow Position Smoothing Speed`**: This will smooth out the hands follow target position over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.
**`Follow Rotation Smoothing Speed`**: This will smooth out the hands follow target rotation over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.
## OpenXR Auto Hand Tracking Grabbing
### **Touch Grab Settings**
**`handTracker`**: Reference to the hand tracker responsible for tracking hand movements and poses
**`releaseGrabDelay`**: The delay in seconds before a grab is released after the grab condition is no longer met
**`fingerTipRadiusMultiplier`**: Multiplier for the radius of the finger tip detection spheres used in touch grabbing
**`useFingerTouchGrabbing`**: Enables or disables grabbing objects by touching them with fingers
**`useFingerTouchReleasing`**: Enables or disables releasing objects by touching them with fingers
**`useTouchHoldingWithHeldPose`**: Enables or disables maintaining a held pose while holding an object with finger touch grabbing
### **Pose Grab Settings**
**`usePoseGrabbing`**: Enables or disables grabbing objects using predefined hand poses
**`minPoseGrabCloseness`**: The minimum closeness required for the hand pose to initiate a grab
**`maxPoseGrabCloseness`**: The maximum closeness allowed for the hand pose to initiate a grab
**`requiredDeltaPoseGrabCloseness`**: The required change in closeness to trigger a grab
### **Pose Release Settings**
**`usePoseRelease`**: Enables or disables releasing objects using predefined hand poses
**`minPoseReleaseOpenness`**: The minimum openness required for the hand pose to initiate a release
**`maxPoseReleaseOpenness`**: The maximum openness allowed for the hand pose to initiate a release
**`requiredDeltaPoseReleaseOpenness`**: The required change in openness to trigger a release
### **Pose Squeeze Settings**
**`usePoseSqueezing`**: Enables or disables squeezing objects using predefined hand poses
**`squeezeUnsqueezeDelay`**: The delay in seconds before a squeeze is unsqueezed after the squeeze condition is no longer met
**`squeezePoseSensitvityMultiplier`**: Multiplier for the sensitivity of the squeezing pose detection
## Raw OpenXR Hand Tracking Input Visuals
These "Right/Left Hand Tracking Visuals" show the default OpenXR hand tracking input before it's sent to the Auto Hand Tracking components.
If hand tracking quality seems low, you can check these visuals to see if it's the hand tracking input that isn't work well or if it's the Auto Hand Tracking that isn't working as expecting
Disable these components to hide the ghost hands
{% endtab %}
{% tab title="MetaXR" %}
## MetaXR Auto Hand Tracking
**`Hand`**: is a reference to which hand this script is driving
**`Up Axis`**: if your skeleton on your hand rig matches the OpenXR hand tracking skeleton, this axis should equal "Up". If your skeleton is different you figure out what your hands' up axis equivalent is on the OpenXR rig. On the robot hand example the yellow up axis is equal to the OpenXR's left (opposite direction of the red arrow)
**`Forward Axis`**: same as previous value, but for your hand's blue forward arrow. In the example below the robot hand's forward axis matches the OpenXR hand's yellow up axis. So it is set to "Up"
This example shows why the "Up/Forward" Axis settings are "Left/Up" on the robot hand tracking
**`Hand Offset`**: Position offset for the hand in local hand tracking space
**`Hand Rotation Offset`**: This offsets the hand relative to it's tracking target. In order for the hand tracking to work the hand model must have correct rotation offset to the tracking hand
**`Hand Pose Smoothing`**: Add a small smooth movement delay when matching the fingers to their target pose. This mainly helps the hand from moving jarringly when in using low confidence poses
**`Follow Position Smoothing Speed`**: This will smooth out the hands follow target position over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.
**`Follow Rotation Smoothing Speed`**: This will smooth out the hands follow target rotation over some. The lower the value the slower smoother/slower the matching. This helps prevent jumping/jittering from lower quality hand tracking input.
## MetaXR Auto Hand Tracking Grabbing
### **Touch Grab Settings**
**`handTracker`**: Reference to the hand tracker responsible for tracking hand movements and poses
**`releaseGrabDelay`**: The delay in seconds before a grab is released after the grab condition is no longer met
**`fingerTipRadiusMultiplier`**: Multiplier for the radius of the finger tip detection spheres used in touch grabbing
**`useFingerTouchGrabbing`**: Enables or disables grabbing objects by touching them with fingers
**`useFingerTouchReleasing`**: Enables or disables releasing objects by touching them with fingers
**`useTouchHoldingWithHeldPose`**: Enables or disables maintaining a held pose while holding an object with finger touch grabbing
### **Pose Grab Settings**
**`usePoseGrabbing`**: Enables or disables grabbing objects using predefined hand poses
**`minPoseGrabCloseness`**: The minimum closeness required for the hand pose to initiate a grab
**`maxPoseGrabCloseness`**: The maximum closeness allowed for the hand pose to initiate a grab
**`requiredDeltaPoseGrabCloseness`**: The required change in closeness to trigger a grab
### **Pose Release Settings**
**`usePoseRelease`**: Enables or disables releasing objects using predefined hand poses
**`minPoseReleaseOpenness`**: The minimum openness required for the hand pose to initiate a release
**`maxPoseReleaseOpenness`**: The maximum openness allowed for the hand pose to initiate a release
**`requiredDeltaPoseReleaseOpenness`**: The required change in openness to trigger a release
### **Pose Squeeze Settings**
**`usePoseSqueezing`**: Enables or disables squeezing objects using predefined hand poses
**`squeezeUnsqueezeDelay`**: The delay in seconds before a squeeze is unsqueezed after the squeeze condition is no longer met
**`squeezePoseSensitvityMultiplier`**: Multiplier for the sensitivity of the squeezing pose detection
If you want to disable the visual hands the OVR includes disable these components found here
{% endtab %}
{% tab title="SteamVR" %}
## IndexHandControllerLink
Grab Fingers Required will trigger grab when the included fingers are bent past the given amount, and release when any fingers are less than this value or when the grab action is triggered/untriggered
{% endtab %}
{% tab title="Oculus Integration (Deprecated)" %}
##
{% hint style="info" %}
The hand tracking Demo can be found within the **Oculus Integration** package. It will be found in Auto Hand/Examples/Scenes/Oculus Integration/Hand Tracking
If you can't find that scene follow the setup guide for [Oculus Integration](https://earnest-robot.gitbook.io/auto-hand-docs/setup#package-setup)
{% endhint %}
This component should go onto your hand along with the OVRHand and OVRSkeleton
Grab Fingers Required will trigger grab when the included fingers are bent past the given amount, and release when any fingers are less than this value
`Thumb, Index, Middle, Ring, Pinky` [finger ](https://earnest-robot.gitbook.io/auto-hand-docs/auto-hand/hand/finger-component)components are found under the hand
`Free Fingers` will allow fingers to move while holding an open (as long as they don't break the grab finger requirements). For example lift the index finger up while holding a soda bottle
`Grab Action` will call the grab event when all the included fingers are bent past the given amount. Release event is called when grab action is no longer valid.
* In this case grab will occur when the index middle and thumb are bent past halfway or 0.5
`Squeeze Action` will trigger the squeeze event when all the included fingers are bent past the given amount.
### Custom Hand Tracking Event Component
Hand Tracking includes an event script which will call Press/Release
* Press is triggered if all the fingers in Finger Bend Past are bent past the bend value, and all the values in Finger Bend Below are below the bend value.
* Release is triggered if press was, but is no longer valid
### Controller Swapping
The OVR Hand Controller Swapper will allow you to automatically swap between controllers and hand tracking. Connect the hand follow targets for hand tracking / controller tracking then connect any objects/scripts that should be disabled on swap
{% endtab %}
{% endtabs %}
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://earnest-robot.gitbook.io/auto-hand-docs/auto-hand/hand-tracking.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
