Human motion can provide structured demonstrations of how a body moves through an action. It does not, by itself, provide the state-action data, forces, torques, tactile signals, or dynamic feasibility required to control a specific robot.
That distinction determines whether a dataset is useful. Human motion supports motion priors, reference trajectories, simulation, behavior coverage, and evaluation. Robot-native demonstrations remain necessary when the learning objective depends on embodiment-specific actions, actuators, sensors, forces, or contact-rich manipulation.
For humanoid systems, human motion is most valuable when it is explicit, temporally consistent, and paired with the information required by the task: joint positions or rotations, root trajectory, contact states, action labels, scene context, or a target-robot representation.
Robot-native data is expensive because it is collected through a physical system with a specific body, sensor stack, and control interface. Hardware must be available. Operators must be trained. Batteries, thermal limits, failures, maintenance, resets, and safe operating envelopes all reduce collection throughput.
The resulting data is also embodiment-specific. A demonstration recorded on one robot encodes that robot's joint axes, degrees of freedom, link lengths, actuator behaviour, observation space, and action space. Moving it to another body is not a file conversion.
Teleoperation is valuable because it can capture robot-native observations and actions close to the intended deployment distribution. But it does not scale like existing human video or offline motion libraries. The bottleneck is not only operator time. It is simultaneous access to the right hardware, environment, objects, sensors, and task setup.
Human motion offers a different source of scale. It records behaviour before committing to one robot. Marker-based motion capture can provide precise human kinematics. Video-derived motion can extract structured movement from a much broader set of performances and environments. Generated motion can fill targeted behaviour categories. None replaces robot-native data. Each adds a different layer.
“Human motion data” can refer to several representations that should not be conflated.
| Representation | What it encodes | What it does not guarantee |
|---|---|---|
| RGB video | Pixels over time, including people, objects, and scene context | Explicit 3D joints, metric scale, robot actions, forces, or hidden-body geometry |
| 2D keypoints | Image-plane joint locations per frame | Depth, global 3D trajectory, full-body orientation, or physical feasibility |
| 3D joint positions | Joint locations in a local or world coordinate frame | A unique set of joint rotations, actuator commands, or valid robot configuration |
| Joint rotations | Orientation of joints relative to a hierarchy or coordinate convention | Equivalent behaviour on a skeleton with different axes, proportions, or degrees of freedom |
| Root trajectory | Global body translation and orientation through the scene | Stable foot contact, balance, or a realisable base trajectory for a robot |
| Contact states | Estimated or measured intervals when feet, hands, or body parts contact surfaces | Contact force, friction, pressure distribution, or successful manipulation |
| Labels and segmentation | Action names, temporal boundaries, task phases, or other metadata | The continuous control signal needed to perform the action |
| Robot state and action | Embodiment-specific observations and commands | Transferability to another robot or behaviour outside the collected policy/interface |
| Force, torque, tactile | Interaction dynamics at joints, actuators, or contact surfaces | Visual semantics or broad behaviour diversity by themselves |
The same clip can produce several of these layers. A training specification should name the required representation, coordinate system, frame rate, skeleton, contact definition, and relationship between observations and actions.
There is no single “robotics dataset.” Teams combine sources based on the learning objective.
| Source | Primary signal | Best use | Main limitation |
|---|---|---|---|
| Robot teleoperation | Robot observations, state, and actions; sometimes force, torque, and contact | Task-aligned demonstrations, manipulation, deployment-specific policy learning | Hardware- and operator-constrained; embodiment-specific |
| Autonomous robot rollouts | Native successes, failures, observations, and actions | On-policy improvement, evaluation, recovery behaviour | Requires a capable policy and physical system; failures can be costly |
| Marker-based human mocap | Precise, temporally clean 3D human kinematics | Motion priors, reference sets, whole-body coordination, evaluation | Studio constraints; no robot actuator signal; requires embodiment transfer |
| Video-derived 3D motion | Estimated human joints, rotations, and trajectories from video | Broad behaviour coverage, existing video conversion, scalable pretraining inputs | Estimated rather than marker-ground-truth; visibility and reconstruction errors |
| Egocentric or internet video | Pixels, actions, objects, environments, and context | Representation learning, world understanding, behaviour discovery | 3D structure and control signals remain implicit unless extracted or paired |
| Simulation or generated motion | Controllable scenarios, synthetic labels, generated reference movement | Coverage, rare behaviours, targeted variation, curriculum construction | Sim-to-real gap; quality depends on physics, assets, policies, and generation method |
Marker-based mocap measures human movement with calibrated cameras and tracked markers. The output can provide accurate trajectories and consistent temporal structure without asking a pose-estimation model to infer every hidden joint from pixels.
Uthana's current licensable studio dataset contains more than 150 hours of full-body, marker-based human motion with finger data, captured in an AAA production environment on a 120-camera system. The data includes action labels, temporal segmentation, root and joint trajectories, foot contacts, and performer/session metadata. The target skeleton and delivery format can be customized for the engagement. Commercial training and evaluation rights are available by agreement.
Explore the studio motion dataset
Video-derived motion trades measurement certainty for behavioural scale. It can convert recorded performances into a common skeletal representation without sending every performer through a capture stage.
The difficult cases are predictable: self-occlusion, motion blur, moving cameras, people leaving the frame, ambiguous depth, loose clothing, object occlusion, and contact that is visually subtle. A data pipeline must measure and filter these failure modes rather than treating every extracted sequence as equivalent to studio capture.
Uthana converts supported single-subject video into animated 3D skeletal motion. The self-serve workflow produces editable FBX, GLB, or BVH motion. A research or data engagement can define a separate schema, alignment method, labels, and validation process around the customer's source video; those fields are not universal self-serve outputs. Explore motion data from video.
Simulation provides controllability. A team can vary trajectories, speeds, bodies, scenes, disturbances, and labels. Generated human motion can target an action or behaviour without recording a new performance.
Both depend on validation. A generated motion may be visually plausible but physically inconsistent. A simulated policy may exploit the simulator. Synthetic variation is useful when its relationship to real capture and deployment data is measured.
A human skeleton and a humanoid robot are not the same kinematic system. They differ in link lengths and body proportions; joint hierarchy and local axes; degrees of freedom; joint limits and reachable workspace; foot and hand geometry; mass distribution and centre of mass; collision geometry; actuator speed, torque, and control bandwidth; and contact behaviour and balance constraints.
Directly copying human joint angles can place the robot in an invalid or unstable configuration. Retargeting instead solves for a target motion that preserves selected properties of the source — such as end-effector paths, orientations, root trajectory, contacts, or pose relationships — while respecting the target's kinematic structure.
A practical pipeline separates five stages:
Uthana can supply human motion, reconstruct motion from video, and map motion to a target embodiment. Policy training, dynamic validation, controller development, and hardware execution remain downstream unless they are explicitly included in a scoped engagement.
Research reinforces this separation. The 2025 paper Retargeting Matters reports that artifacts introduced during human-to-humanoid retargeting can reduce policy robustness, particularly for dynamic or long sequences. The 2026 MOSAIC system combines a multi-source motion bank, retargeting, reinforcement-learning-based tracking, and interface-specific adaptation — multiple layers, not one conversion step.
Human motion supplies examples of timing, coordination, posture changes, and whole-body behaviour. These references can define what a motion-tracking policy should imitate or help a model learn a structured motion space.
The usefulness depends on coverage. A large collection of similar walks is not equivalent to broad behaviour data. Teams should inspect action diversity, transitions, contacts, performer diversity, and the distribution of clip lengths — not only total hours.
Walking, running, turning, stopping, crouching, reaching, balancing, and recovering involve coordinated motion across the body. Human reference motion can support locomotion priors or tracking objectives when it is retargeted to the target morphology and validated under the robot's dynamics.
Simulated environments need moving people and humanoid agents that respond to a requested direction, speed, and route rather than replaying a fixed clip. Motion libraries and controllable locomotion can provide behaviour primitives for these agents.
Uthana generates flat-ground locomotion clips from direction, speed, stride count, and style inputs. The output can be downloaded as FBX, GLB, or BVH and packaged by the customer for its downstream workflow. Uthana is not naming tested simulator integrations or presenting locomotion as environment-aware robot control at launch. Explore locomotion.
A team can identify an underrepresented action, record it with marker-based capture, extract examples from licensed video, or generate candidate motions. Labels and temporal segmentation then make the motion searchable and trainable.
This is most useful when the acquisition plan starts from a training specification. “More motion” is not a specification. Define actions, transitions, body states, object context, contact phases, frame rate, representation, and acceptable error.
Ground-truth motion can serve as a reference for pose estimation, motion generation, retargeting, and tracking evaluation. The evaluation set should be isolated from training data and should include hard cases that reflect deployment: fast motion, turns, occlusion, contacts, unusual proportions, and long sequences.
Human motion data should not be presented as a substitute for every robotics signal.
A 3D hand path does not reveal grip force, contact pressure, slip, object mass, friction, or tactile response. Contact labels can indicate when interaction occurs, but not the forces that make it successful.
Human joint rotations are not robot motor commands. Even after retargeting, a sequence of target joint positions is not identical to torque commands, actuator targets, or a learned action policy.
Mass distribution, actuator limits, compliance, latency, and control frequency shape what a robot can execute. A kinematically valid pose sequence can still fail under dynamics.
Motion alone may show what the body did without fully representing why it did it, which object state changed, whether the task succeeded, or which observation triggered the next action. Task learning often needs synchronised video, object pose, language, force, tactile, or outcome labels.
Studio capture is clean because the environment is controlled. Deployment is not. Human motion should be combined with robot rollouts and task-specific data when the policy must tolerate sensor noise, surface variation, payload changes, disturbances, and recovery.
| Motion source | Strongest property | Typical risk | Best role |
|---|---|---|---|
| Marker-based studio capture | Spatial and temporal ground truth under a controlled setup | Limited environments and costly capture expansion | Evaluation, precise priors, high-confidence training subsets |
| Video-derived motion | Access to broad behaviour and existing footage | Reconstruction error, occlusion, uncertain contacts | Scale, diversity, context-rich coverage after QA |
| Generated motion | Targeted actions and controllable variation | Model bias, implausible contacts, limited exactness | Coverage expansion, simulation primitives, candidate generation |
| Robot-native data | Direct alignment with the target system | Hardware cost and embodiment specificity | Policy learning, manipulation, deployment validation |
A robust program can use studio data as an accuracy anchor, video-derived motion for breadth, generated motion for controlled coverage, and robot-native data for the final control problem.
Uthana provides a documented CSV output path for Unitree G1. A Uthana motion ID is converted into a G1 robot trajectory and downloaded through an authenticated HTTP request. The customer then loads the CSV into its own playback or controller workflow.
The CSV output is target-specific kinematic motion. It does not by itself provide forces, torques, tactile state, environment state, controller logic, or proof of safe hardware execution.
View the Unitree G1 output workflow
Downstream, the robotics team validates joint limits, collision, balance, contacts, and dynamic tracking in its simulator and controller. If the clip is used for policy learning, the team defines the observation space, action space, reward or objective, and deployment validation.
Ask for a sample that matches the proposed delivery — not only a highlight reel.
Human motion data is a time-based representation of how a human body moves. It can include 3D joint positions, joint rotations, root trajectory, contact states, labels, and temporal segmentation. It is useful as demonstration or reference data, but it is not automatically robot state-action data.
Human motion describes the performer's body. Teleoperation data can include the target robot's observations, state, and actions while an operator controls it. Teleoperation is closer to the robot's control problem; human motion can offer broader behaviour coverage before committing to one embodiment.
Yes, as reference or demonstration data in workflows such as motion tracking, imitation, priors, simulation, and evaluation. It usually must be retargeted to the target robot and validated for kinematic and dynamic feasibility. Some objectives also require robot-native actions, forces, tactile data, or rollouts.
Humans and humanoid robots have different proportions, joint axes, degrees of freedom, limits, link geometry, and dynamics. Retargeting maps selected properties of the source motion into a robot-specific joint representation rather than copying angles directly.
Most skeletal motion does not contain actuator commands, motor torques, tactile state, grip force, ground-reaction force, object properties, or proof of dynamic feasibility. These signals must be measured, simulated, inferred, or collected from the robot depending on the task.
Use marker-based capture when precise trajectories, repeatability, and ground-truth-quality evaluation matter. Use video-derived motion when behaviour breadth, existing footage, or acquisition scale matters. Many programs use captured motion as an accuracy anchor and video-derived motion for coverage.
Not necessarily. A retargeted sequence can be kinematically plausible while violating balance, collision, torque, velocity, or contact constraints. Validate it in the target simulator and control stack before hardware execution.
Start with the representation, behavior coverage, embodiment, and rights your program requires. Uthana can combine studio capture, video extraction, labeling, and robot retargeting into a dataset built to that specification.
Marker-based studio motion captured on a 120-camera system, licensed for training and evaluation.
Explore the studio datasetConvert human motion into joint representations built for a target robot embodiment.
Explore robot retargetingHuman demonstrations as a source of robot training data, scoped to your embodiment.
Explore robot trainingStart generating motion today, or work with us on a custom dataset, embodiment system, or simulation program.
