Inside NVIDIA’s IsaacTeleop: From Hand and Controller Tracking to Robot Actions with the Graph-Based Retargeting Engine
NVIDIA IsaacTeleop tutorial maps XR hand and controller tracking to robot actions via a graph retargeter.
MarkTechPost walks through NVIDIA IsaacTeleop 1.4.145, which turns XR hand tracking and motion-controller input into commands for simulated and real robots. The retargeters-lite extra supplies a pure-Python graph engine over FlatBuffer schema types, while separate modules wrap OpenXR and CloudXR device I/O. The tutorial builds synthetic HandInput tensors for 26 OpenXR joints and 14-slot ControllerInput groups, then drives built-in gripper and SE(3) retargeters. A composed graph emits one action vector per step, applies a world-frame transform, and supports pause, kill, and parameter tuning that persists across restarts.
- IsaacTeleop 1.4.145 maps XR hand and controller input to robot commands.
- Tutorial uses synthetic NumPy tensors without a headset or simulator.
- HandInput covers 26 OpenXR joints; ControllerInput has fourteen slots.
- A graph emits one action vector per step with tunable retargeter parameters.
- Vendors
- NVIDIA
- Products
- IsaacTeleop
- Organizations
- NVIDIA
From the original article
10 min read at marktechpost.com
Learn how NVIDIA IsaacTeleop turns XR hand tracking and motion controller input into robot commands using a pure Python retargeting engine and NumPy. The post Inside NVIDIA’s IsaacTeleop: From Hand and Controller Tracking to Robot Actions with the Graph-Based Retargeting Engine appeared first on MarkTechPost.
By Asif Razzaq · marktechpost.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.