⚡ DWPose Video Batch Inference

High-throughput, multi-GPU batch inference pipeline for 2D Whole-Body Pose Estimation (133 Keypoints)

📖 Related Paper: This repository is utilized as a key whole-body pose estimation tool in the paper:
"PLNet-12: A Vision-Language Benchmark for Zero-Shot Physical Literacy Analysis Across 12 Fundamental Movements" (AJCAI 2025).

⭐ Please give a star if you find this project helpful or use it in your research!

---

📑 Table of Contents

• ✨ Key Features
• 🔄 Inference Architecture
• 📁 Project Structure
• ⚙️ Installation
• 📥 Model Weights Download
• 🚀 Quick Start & Usage
  • Command-Line Interface (CLI)
  • CLI Arguments Reference
  • Python API Usage
• ⚡ Hardware & Batch Sizing
• 📊 Output Specifications
  • 1. Pose Sequence (N, 406) & Keypoint Visualization
  • 2. Bounding Box (N, 1, 4)
  • 3. Video Metadata
• 📜 License, Citation & Acknowledgments

---

✨ Key Features

• 🚀 Maximized Batch Throughput: Decoupled batching stages for human bounding box detection (YOLOX) and whole-body pose estimation (DWPose), delivering massive speedups over single-frame execution.
• 🧍 133 Whole-Body Keypoints: Full COCO-WholeBody representation covering 17 body joints, 6 foot points, 68 facial landmarks, and 42 hand keypoints.
• ⚡ Multi-Process Video Decoding: Automatic parallel CPU chunk decoding for long video clips (>4500 frames) with aspect-ratio preserving letterbox padding.
• 🎛️ Dynamic Downsampling & Trimming: Built-in support for frame rate decimation (extraction_ratio) and duration clipping (max_length).
• 🎬 Flexible Visual Rendering: Save rich animated skeleton overlays directly as .mp4 video files or sequential high-resolution .png frames.

---

🔄 Inference Architecture

flowchart TD
    subgraph Input ["🎬 Video Input & Decoding"]
        V["Input Video File (.mp4 / .mov)"] --> VD["VideoDataset (Parallel CPU Decode & Letterbox Padding)"]
    end

    subgraph Detection ["🔍 Stage 1: Human Detection"]
        VD -->|"Image Batches (batch_size_1)"| YOLOX["YOLOX-L Detector (GPU)"]
        YOLOX --> NMS["Bounding Box NMS & Area Sorting"]
    end

    subgraph PoseEstimation ["🤸 Stage 2: Whole-Body Pose Estimation"]
        NMS -->|"BBox + Image Batches (batch_size_2)"| DW["DWPose-L Estimator (GPU)"]
        DW --> DWProc["DWProcessor (Coordinate Normalization & 406-D Assembly)"]
    end

    subgraph Output ["📊 Output Generation"]
        DWProc --> PO["Pose Sequences (N, 406) & BBoxes (N, 1, 4)"]
        DWProc -->|"Render Overlays (vis_res)"| VR["Visualized Video (.mp4) / Frames (.png)"]
    end

    style Input fill:#e1f5fe,stroke:#0288d1,stroke-width:2px;
    style Detection fill:#fff3e0,stroke:#f57c00,stroke-width:2px;
    style PoseEstimation fill:#e8f5e9,stroke:#388e3c,stroke-width:2px;
    style Output fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px;

Loading

---

📁 Project Structure

DWPose-Video-Batch-Inference/
├── asset/                               # Project documentation assets
│   └── Fig2_anno.png                    # COCO-WholeBody 133 keypoint topology annotation
├── DWPose_usage/                        # DWPose & YOLOX inference package
│   ├── __init__.py                      # DWProcessor wrapper and skeleton drawing routines
│   ├── wholebody.py                     # Wholebody detector & pose estimation coordinator
│   ├── util.py                          # Rendering helpers (body, face, hand, foot) & math transforms
│   ├── inference_detector.py            # Batched human detector inference for MMDetection
│   ├── inference_topdown.py             # Batched top-down pose estimator inference for MMPose
│   ├── mmpose_inferencer.py             # MMPose inferencer adapter
│   ├── pose2d_inferencer.py             # 2D pose inferencer module
│   ├── dwpose_config/                   # DWPose configuration and checkpoint directory
│   │   ├── dwpose-l_384x288.py          # DWPose-L architecture configuration
│   │   └── dw-ll_ucoco_384.pth          # (Place DWPose checkpoint here)
│   └── yolox_config/                    # YOLOX detector configuration and checkpoint directory
│       ├── yolox_l_8xb8-300e_coco.py    # YOLOX-L detector configuration
│       └── yolox_l_8x8_300e_coco_...pth # (Place YOLOX checkpoint here)
├── video_pose_estimation.py             # Main batch inference script with CLI argument parser
├── LICENSE                              # License file
└── README.md                            # Project documentation

Component Breakdown

• video_pose_estimation.py: The main entry point. Contains VideoDataset (dataset handler with multi-threaded decoding and letterbox padding), video_batch_inference(), video_pose_estimation(), and the command-line interface with argument parsing (parse_args).
• DWPose_usage/wholebody.py: The Wholebody class initializes the YOLOX human detector and DWPose pose estimator, extracts non-maximum suppressed (NMS) bounding boxes, and predicts whole-body keypoint coordinates.
• DWPose_usage/__init__.py: Contains DWProcessor which normalizes coordinates, builds the 406-dimensional pose sequence vectors, and handles pose rendering overlays.
• DWPose_usage/util.py: Mathematical utilities and specialized OpenCV drawing functions for limbs, facial landmarks, hands, and feet.
• DWPose_usage/inference_detector.py & DWPose_usage/inference_topdown.py: Batched data pipeline forward wrappers for MMDetection and MMPose.

---

⚙️ Installation

1. Create and Activate Conda Environment

conda create --name dwpose python=3.8 -y
conda activate dwpose

2. Install PyTorch with CUDA Support

Install PyTorch matching your CUDA version (see PyTorch Official Guide):

# Example for CUDA 11.8 / 12.1
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118

3. Install OpenMMLab Packages

pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.1"
mim install "mmdet>=3.1.0"
mim install "mmpose>=1.1.0"
pip install opencv-python Pillow matplotlib

Important

Compatibility Version Patch: If using MMCV >= 2.2.0, update the maximum version check in MMDetection: Open <conda_path>/envs/dwpose/lib/python3.8/site-packages/mmdet/__init__.py and modify:

# Change from:
mmcv_maximum_version = '2.2.0'
# To:
mmcv_maximum_version = '2.2.1'

---

📥 Model Weights Download

Download the pretrained weights and place them into their respective directories:

Model	Checkpoint File	Download Link	Target Destination
YOLOX-L	yolox_l_8x8_300e_coco_20211126_140236-d3bd2b23.pth	Download (OpenMMLab)	DWPose_usage/yolox_config/
DWPose-L	dw-ll_ucoco_384.pth	Download (HuggingFace)	DWPose_usage/dwpose_config/

Quick Download Commands:

# Download YOLOX-L weights
wget -P DWPose_usage/yolox_config/ https://download.openmmlab.com/mmdetection/v2.0/yolox/yolox_l_8x8_300e_coco/yolox_l_8x8_300e_coco_20211126_140236-d3bd2b23.pth

# Download DWPose weights
wget -P DWPose_usage/dwpose_config/ https://huggingface.co/yzd-v/DWPose/resolve/main/dw-ll_ucoco_384.pth

---

🚀 Quick Start & Usage

Command-Line Interface (CLI)

# 1. Basic pose estimation with video rendering
python video_pose_estimation.py --video_path ./demo_video.mp4 --save_to_video_path ./output/

# 2. Save both rendered video and individual frame images
python video_pose_estimation.py \
    --video_path ./demo_video.mp4 \
    --save_to_video_path ./output/ \
    --save_to_images_path ./output/frames/

# 3. High-throughput execution on NVIDIA RTX 4090 (24GB VRAM)
python video_pose_estimation.py -v ./demo_video.mp4 -sv ./output/ -b1 185 -b2 375

# 4. Frame downsampling (process 1 frame every 3 frames)
python video_pose_estimation.py -v ./demo_video.mp4 -sv ./output/ -r 3

# 5. Duration limit (process only first 15 seconds)
python video_pose_estimation.py -v ./demo_video.mp4 -sv ./output/ -m 15.0

---

CLI Arguments Reference

Argument	Short Flag	Type	Default	Description
--video_path	-v	str	./blob_...mov	Path to the input video file (e.g. .mp4, .mov, .avi).
--save_to_video_path	-sv	str	./	Directory path to save visualized video. Pass "" or 'None' to disable.
--save_to_images_path	-si	str	None	Directory path to save visualized frame images. Pass None to disable.
--batch_size_1	-b1	int	185	Batch size for human detection (YOLOX).
--batch_size_2	-b2	int	375	Batch size for whole-body pose estimation (DWPose).
--extraction_ratio	-r	int	0	Extraction ratio: >1 extracts 1 every X frames; <-1 extracts |X| frames per frame; 0 processes all frames. Cannot be 1.
--max_length	-m	float	0.0	Maximum video duration in seconds to process (0 processes full video).

---

Python API Usage

from video_pose_estimation import video_pose_estimation

# Run batch pose estimation
pose_sequences, bboxes_list, video_meta = video_pose_estimation(
    video_path="path/to/your_video.mp4",
    save_to_video_path="./output/",         # Directory to save rendered video (or None)
    save_to_images_path="./output/frames/",  # Directory to save rendered frame images (or None)
    batch_size_1=185,                        # Detection batch size (YOLOX)
    batch_size_2=375,                        # Pose estimation batch size (DWPose)
    extraction_ratio=0,                      # 0 to process all frames
    max_length=0                             # 0 to process entire duration
)

# Inspect outputs
print("Total Frames Processed :", video_meta["frame"])
print("Video Resolution       :", f"{video_meta['W']}x{video_meta['H']}")
print("Pose Sequence Array Dim:", pose_sequences.shape)  # (N, 406)
print("Bounding Boxes Count   :", len(bboxes_list))      # N

---

⚡ Hardware & Batch Sizing

Optimize memory and GPU throughput by selecting appropriate batch sizes:

GPU Model	VRAM	Detector Batch Size (batch_size_1)	Pose Estimator Batch Size (batch_size_2)
NVIDIA A100	80 GB	600	1220
NVIDIA RTX 4090	24 GB	185	375
NVIDIA RTX 3090 / 4080	16–24 GB	150	300
NVIDIA Tesla T4	16 GB	115	240
NVIDIA RTX 3060 / 4060	8–12 GB	50	100

---

📊 Output Specifications

1. Pose Sequence (N, 406) & Keypoint Visualization

For each frame $i \in [0, N-1]$, pose_sequences[i] is a 406-dimensional vector:

$$\text{Vector Length} = 7 + (133 \times 3) = 406$$

Index:  [0 ............. 6] [7 .................................... 405]
Data :  [ Bounding Box + Score ] [ 133 Keypoints: (x, y, confidence) x 133 ]

🖼️ COCO-WholeBody 133 Keypoint Topology & Name Reference

The visual definition and keypoint ID numbering ($1 \sim 133$) are illustrated below:

Figure: COCO-WholeBody 133 Keypoints Topology, ID mapping (1–133), and corresponding anatomical names.

📐 Index Calculation & Format Breakdown

• Bounding Box & Score (Index 0 ~ 6):

  • [0, 0, x1, y1, x2, y2, score]
  • (x1, y1): Top-left corner of the detected person bounding box (pixels).
  • (x2, y2): Bottom-right corner of the detected person bounding box (pixels).
  • score: Confidence score of the bounding box detection.

• 133 Whole-Body Keypoints (Index 7 ~ 405):

  • Each keypoint contains [x_kpt, y_kpt, confidence_score].
  • Coordinates (x_kpt, y_kpt) are normalized to [0.0, 1.0] relative to frame width $W$ and height $H$.
  • Conversion to absolute pixel coordinates: $$X_{\text{pixel}} = x_{\text{kpt}} \times W, \quad Y_{\text{pixel}} = y_{\text{kpt}} \times H$$
  • Index Mapping Formula: For the $x$-th keypoint shown in the figure above ($x \in [1, 133]$), its corresponding elements in the pose_sequences array (0-indexed) are: $$\text{X-coordinate: } 3x + 4, \quad \text{Y-coordinate: } 3x + 5, \quad \text{Confidence Score: } 3x + 6$$ (e.g., Keypoint 1: Nose $\rightarrow$ indices [7, 8, 9]; Keypoint 133: Right Hand Tip $\rightarrow$ indices [403, 404, 405])

🔍 Click to view Keypoint Index Partitions & COCO-WholeBody Group Mapping

Keypoint Group	Keypoint IDs in Figure	Keypoint Count	Array Index Range	Details / Keypoint Names
Body Joints	$1 \sim 17$	17	7 ~ 57	Nose (1), Eyes (2,3), Ears (4,5), Shoulders (6,7), Elbows (8,9), Wrists (10,11), Hips (12,13), Knees (14,15), Ankles (16,17)
Foot Points	$18 \sim 23$	6	58 ~ 75	Big toe, small toe, heel for left and right feet
Face Landmarks	$24 \sim 91$	68	76 ~ 279	Face contour, eyebrows, nasal bridge, nose tip, eyes, and inner/outer lips
Hand Keypoints	$92 \sim 133$	42	280 ~ 405	21 keypoints for left hand ($92 \sim 112$) + 21 keypoints for right hand ($113 \sim 133$)

For additional topology details, refer to the COCO-WholeBody Dataset Specification.

---

2. Bounding Box (N, 1, 4)

• List of length $N$, with each frame entry formatted as [[x1, y1, x2, y2]] in absolute pixel coordinates.

---

3. Video Metadata

{
    "fps": 30,         # Video frames per second
    "frame": 450,      # Total processed frames
    "H": 640,          # Video height (pixels)
    "W": 640,          # Video width (pixels)
    "channel": 3,      # Color channels (BGR)
    "seconds": 15.0    # Processed duration in seconds
}

---

📜 License, Citation & Acknowledgments

This project is released under the Apache 2.0 License.

📚 Citation

If you find this repository or tool helpful in your research, please consider citing:

@inproceedings{guo2025plnet,
  title={Plnet-12: A vision-language benchmark for zero-shot physical literacy analysis across 12 fundamental movements},
  author={Guo, Tianchen and Logan, Peter Anthony and Wackwitz, Thomas and Martin, David},
  booktitle={Australasian Joint Conference on Artificial Intelligence},
  pages={242--254},
  year={2025},
  organization={Springer}
}

🙏 Acknowledgments

Special thanks and acknowledgment to the underlying open-source projects:

• DWPose / Effective Whole-body Pose Estimation with Two-stage Distillation
• MMPose & MMDetection by OpenMMLab
• COCO-WholeBody