FlowHMR: Free AI Video-to-3D Human Motion Capture

FlowHMR turns footage from a single camera into global 3D human motion.

Upload a clip, select a person if several appear, and watch the reconstructed movement alongside the original video.

You can export the result as NPZ or FBX motion data, or process your own videos with the local inference tools.

FlowHMR Code

FlowHMR Playground

Key Features

Reconstruct global 3D body movement and trajectories from single-camera video.
Preview motion alongside synchronized source-video playback.
Select a person to track when a clip contains multiple people.
Download reconstructed motion as NPZ or FBX.
Run the 460-million-parameter base or physics-trained model on a local GPU.
Process video files or directories through a CLI.
Handle stationary and moving cameras with configurable camera estimation.
Run a self-hosted webUI and access the training and synthetic-data tools.

Try FlowHMR in Your Browser

1. Open the FlowHMR Hugging Face, upload a video, and click Generate motion.

2. For footage containing several people, choose the person whose movement you want to reconstruct and continue generation.

3. The synchronized viewer lets you inspect body poses and movement through the scene as the video plays. You can rotate and zoom the 3D view, and a ground trajectory traces the person’s movement up to the current frame.

FlowHMR Free AI Video-to-3D Human Motion Controls

4. Download NPZ motion data for processing in Python or FBX for an animation pipeline. The exported motion contains body articulation and global translation for downstream animation or analysis.

How FlowHMR Reconstructs 3D Motion

A single camera records movement in two dimensions, leaving depth, occluded joints, and world-space movement to be estimated. FlowHMR generates a sequence of articulated body poses and a global trajectory conditioned on the video.

Its inference pipeline uses YOLOX for person detection and tracking, SAM 3D Body for body features, and VGGT-Omega to estimate camera movement when the footage was recorded with a moving camera.

The FlowHMR model uses these inputs to reconstruct SMPL-H motion, including body and hand poses plus translation through the scene. Footage from a stationary camera can use the static-camera option to skip VGGT estimation.

The project releases two 0.46-billion-parameter model checkpoints:

Model	Training	PHC+ tracking success on Wild-4K
`flowhmr_base`	Video-conditioned flow matching	78.79%
`flowhmr_latest`	Flow matching followed by GRPO with a physics-based reward	82.47%

Run FlowHMR Locally

The V2M environment supports local inference, the self-hosted browser demo, and base-model training.

Install the environment

Clone the repository with its submodules and install the Python dependencies:

git clone --recursive https://github.com/flowhmr/flowhmr.git FlowHMR
cd FlowHMR
conda create -n flowhmr python=3.10 -y
conda activate flowhmr
pip install -r requirements.txt -r tools/web/requirements.txt
pip install --no-build-isolation --no-deps -e third_party/YOLOX

Download the model assets

FlowHMR uses several perception and body-model assets. Request Hugging Face access to SAM 3D Body and VGGT-Omega, then authenticate in your terminal. Download the YOLOX and perception checkpoints into the locations expected by the repository:

huggingface-cli login
mkdir -p ckpts/yolox
wget -O ckpts/yolox/yolox_l.pth \
  https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_l.pth
huggingface-cli download facebook/VGGT-Omega vggt_omega_1b_512.pt \
  --local-dir ckpts/vggt-omega
huggingface-cli download facebook/sam-3d-body-dinov3 \
  --local-dir ckpts/sam-3d-body-dinov3

Download the SMPL-H neutral body model after accepting its license through MANO. The required file path is assets/body_models/smplh/neutral/model.npz. Add the project’s public joint regressor:

wget -O assets/body_models/smpl_neutral_J_regressor.pt \
  https://raw.githubusercontent.com/zju3dv/GVHMR/main/hmr4d/utils/body_model/smpl_neutral_J_regressor.pt

Download the recommended physics-trained FlowHMR checkpoint:

huggingface-cli download fafsaf1/flowhmr-0.46B \
  --include "flowhmr_latest/*" "config.json" \
  --local-dir checkpoints

The checkpoint is loaded from checkpoints/flowhmr_latest/flowhmr_latest.ckpt, with its accompanying config.yml in that directory.

Generate your first motion file

Run the bundled example video:

python run_v2m_demo_generation.py assets/example/example.mp4

The documented successful run writes output/example/flowhmr_latest.npz, with poses of shape (T, 52, 3) and global trans of shape (T, 3). To process your own clip, pass its filename or a directory of videos:

python run_v2m_demo_generation.py my-video.mp4

For a stationary camera, add --static_camera. For moving-camera footage, --vggt_interval 10 estimates the camera at a wider interval to reduce GPU use.

Local inference saves NPZ by default. To add FBX export, install the optional FBX SDK package in your Python environment:

pip install fbxsdkpy==2020.1.post2 \
  --extra-index-url https://gitlab.inria.fr/api/v4/projects/18692/packages/pypi/simple

Optional --postprocess uses RGBMocap and Kimodo to correct ground contacts and anatomical motion after the packages are installed.

Open the self-hosted web UI

With the V2M environment and checkpoints installed, launch the local browser demo:

python app.py --port 8080 --device cuda:0

Open http://localhost:8080, upload a video, and preview the reconstructed motion. The web UI lets you choose a tracked person, adjust camera estimation and sampling, and revisit previous jobs. Generated results are stored under output/web_jobs/.

Pros

Reconstructs world-space human movement from a single video, with physics-aware model training.
Offers an online preview, NPZ/FBX downloads, and local inference tools.
Includes the source code for motion training, physics-based evaluation, and synthetic-data generation.

Cons

The model weights restrict commercial use.
Local inference requires a capable CUDA GPU and several additional model assets.
Character animation and humanoid deployment depend on retargeting tools.