Welcome to the exciting world of motion capture! In this article, we delve into the process of utilizing the official PyTorch implementation of the LEMO framework, demonstrated in the 2021 ICCV paper “Learning Motion Priors for 4D Human Body Capture in 3D Scenes.” This guide will be user-friendly, breaking down complex tasks into digestible steps, and will include troubleshooting tips to ensure smooth sailing!

What You Need to Get Started

Before diving into the installation and code execution, make sure you have the following requirements:

• Operating System: Ubuntu 18.04
• Python Version: 3.8.5
• CUDA Version: 10.0

Installation Steps

To get the framework up and running, follow these steps:

1. Download the following models:
   • SMPL-X models: version 1.0
   • VPoser model: version 1.0
2. If you plan to use the temporal fitting code for the PROX dataset, you need to install additional packages:
   • Chamfer Pytorch
   • Mesh Packages (optional)
   • PyTorch Mesh self-intersection (optional)
3. Download the part segmentation file: smplx_parts_segm.pkl
4. Install other dependencies by running:

   pip install -r requirements.txt

Note: Be aware that different versions of SMPL-X and VPoser may affect the results you achieve.

Datasets to Use

The LEMO framework works best with the following datasets:

• PROX
• AMASS

Understanding Motion Prior Training Through Analogy

Imagine you’re a choreographer deciding how to create a dance routine. You observe various styles and combine them into a unique performance. This is similar to how motion smoothness and infilling prior models are trained. The models rely on existing motion data (like the styles) and refine their performance through training. You’d use:

python train_smooth_prior.py --amass_dir PATH_TO_AMASS --body_model_path PATH_TO_SMPLX_MODELS --body_mode=global_markers

And for the infilling prior:

python train_infill_prior.py --amass_dir PATH_TO_AMASS --body_model_path PATH_TO_SMPLX_MODELS --body_mode=local_markers_4chan

Fitting on AMASS and PROX Datasets

Fitting the models consists of two stages on the AMASS dataset:

1. Stage 1: Per-frame fitting
   • Utilize motion infilling prior
   • Execute:

     python opt_amass_perframe.py --amass_dir=PATH_TO_AMASS --body_model_path=PATH_TO_SMPLX_MODELS --body_mode=local_markers_4chan --dataset_name=TotalCapture --start=0 --end=100 --step=20 --save_dir=PATH_TO_SAVE_RESULTS

2. Stage 2: Temporal fitting
   • Utilize motion smoothness and infilling prior
   • Run:

     python opt_amass_tempt.py --amass_dir=PATH_TO_AMASS --body_model_path=PATH_TO_SMPLX_MODELS --body_mode=local_markers_4chan --dataset_name=TotalCapture --start=0 --end=100 --step=20 --perframe_res_dir=PATH_TO_PERFRAME_RESULTS --save_dir=PATH_TO_SAVE_RESULTS

Ensure that all parameters such as start, end, and step are consistent across stages.

Visualization of Fitting Results

To visualize your fitted results, run:

python vis_opt_amass.py --body_model_path=PATH_TO_SMPLX_MODELS --dataset_name=TotalCapture --start=0 --end=100 --step=20 --load_dir=PATH_TO_FITTED_RESULTS

Troubleshooting Tips

If you encounter issues, consider the following troubleshooting ideas:

• Check that all paths are correctly specified, especially for models and datasets.
• Ensure that you are using compatible versions of the models.
• Look for error messages during installation or execution to identify specific problems.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.