Skip to content

Latest commit

 

History

History
90 lines (74 loc) · 4.86 KB

POSE_README.md

File metadata and controls

90 lines (74 loc) · 4.86 KB
Raw

Sapiens: 2D Human Pose Estimation

Model Zoo

We provide 4 models of varying size. Sapiens-0.3B, Sapiens-0.6B, Sapiens-1B, Sapiens-2B. In general, performance improves with increasing the model size.

Body: 17 Keypoints

Best for general in-the-wild scenarios with body keypoints only, adhering to the COCO keypoint format.
Please download the models from hugging-face-pose.

Model Checkpoint Path
Sapiens-0.3B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.3b/sapiens_0.3b_coco_best_coco_AP_796.pth
Sapiens-0.6B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.6b/sapiens_0.6b_coco_best_coco_AP_812.pth
Sapiens-1B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_1b/sapiens_1b_coco_best_coco_AP_821.pth
Sapiens-2B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_2b/sapiens_2b_coco_best_coco_AP_822.pth

Body + Face + Hands + Feet: 133 Keypoints

Offers second-best generalization with body, face, hands, and feet keypoints, following the COCO-WholeBody keypoint format.
Please download the models from hugging-face-pose.

Model Checkpoint Path
Sapiens-0.3B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.3b/sapiens_0.3b_coco_wholebody_best_coco_wholebody_AP_620.pth
Sapiens-0.6B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.6b/sapiens_0.6b_coco_wholebody_best_coco_wholebody_AP_695.pth
Sapiens-1B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_1b/sapiens_1b_coco_wholebody_best_coco_wholebody_AP_727.pth
Sapiens-2B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_2b/sapiens_2b_coco_wholebody_best_coco_wholebody_AP_745.pth

Body + Dense Face + Hands + Feet: 308 Keypoints

The highest number of keypoints predictor. Detailed 274 face keypoints. Following the Sociopticon keypoint format. Please download the models from hugging-face.

Model Checkpoint Path
Sapiens-0.3B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.3b/sapiens_0.3b_goliath_best_goliath_AP_573.pth
Sapiens-0.6B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.6b/sapiens_0.6b_goliath_best_goliath_AP_609.pth
Sapiens-1B $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_1b/sapiens_1b_goliath_best_goliath_AP_639.pth

Inference Guide

Navigate to the script directory and launch the inference scripts for your desired keypoint configuration.

Standard Estimation

  • Navigate to your script directory: 
    cd $SAPIENS_ROOT/pose/scripts/demo/local
  • For 17 keypoints estimation (uncomment your model config line for inference): 
    ./keypoints17.sh
  • For 133 keypoints estimation (uncomment your model config line for inference): 
    ./keypoints133.sh
  • For 308 keypoints estimation (uncomment your model config line for inference): 
    ./keypoints308.sh

Define INPUT for your image directory and OUTPUT for results. Visualization and keypoints in JSON format are saved to OUTPUT. Customize LINE_THICKNESS, RADIUS, and KPT_THRES as needed. Adjust JOBS_PER_GPU, TOTAL_GPUS and VALID_GPU_IDS for multi-GPU configurations.

Keypoints 17 Keypoints 133 Keypoints 308

ControlNet Compatible Output

Generate OpenPose skeletons on a black background, compatible with ControlNet.

  • Navigate to your script directory: 
    cd $SAPIENS_ROOT/pose/scripts/demo/local
  • For 17 keypoints estimation (uncomment your model config line for inference): 
    ./keypoints17_openpose.sh
  • For 133 keypoints estimation (uncomment your model config line for inference): 
    ./keypoints133_openpose.sh

Customization options as mentioned above are applicable here as well.

Keypoints 17 Keypoints 133