Welcome to the world of real-time action recognition! In this guide, we’ll explore how to implement a novel system that can recognize up to 9 different actions performed by multiple people using the human skeleton as the primary data source. This project showcases a fun and practical application of computer vision and machine learning. Let’s dive right in!

Highlights of the Project

• Allows recognition of 9 actions.
• Supports up to 5 individuals performing actions simultaneously.
• Utilizes a real-time and multi-frame recognition algorithm.

Table of Contents

• 1. Algorithm
• 2. Install Dependency (OpenPose)
• 3. Program Structure
• 4. How to Run: Inference
• 5. Training Data
• 6. How to Run: Training
• 7. Result and Performance

1. Algorithm

In this project, you will be recognizing actions such as stand, walk, run, jump, sit, squat, kick, punch, and wave. Imagine capturing the movements of a group of people in a square; each participant is a distinct actor performing a unique dance, whose choreography is dictated by body joint movements recorded in real-time.

The process involves a few key steps:

• Extract joint positions using OpenPose.
• Track each individual in the scene by using Euclidean distance between skeletal joint positions.
• Handle missing joint data based on prior frames.
• Augment the data by adding noise to joint positions.
• Use windowed data to extract features for classification and apply DNNs.

2. Install Dependency (OpenPose)

This project requires Python 3.6. Below are the essential steps to set up OpenPose:

• Clone the repository from tf-pose-estimation.
• Download the pretrained models (already included).
• Install the necessary libraries.

Make sure to follow the installation steps accurately to avoid compatibility errors.

3. Program Structure

The overall structure of the program can be visualized as a sequence of functions working together like a well-oiled machine. Each primary script represents a cog that plays its part:

• src/s1_get_skeletons_from_training_imgs.py
• src/s2_put_skeleton_txts_to_a_single_txt.py
• src/s3_preprocess_features.py
• src/s4_train.py
• src/s5_test.py

4. How to Run: Inference

Before you start testing, here’s how to set up the inference: Choose input from a video file, a folder of images, or a web camera.

• To test a video file:

bash
python srcs5_test.py --model_path modeltrained_classifier.pickle --data_type video --data_path data_test/exercise.avi --output_folder output

• To test a folder of images:

bash
python srcs5_test.py --model_path modeltrained_classifier.pickle --data_type folder --data_path data_test/apple --output_folder output

• To test via web camera:

bash
python srcs5_test.py --model_path modeltrained_classifier.pickle --data_type webcam --data_path 0 --output_folder output

5. Training Data

To set up your training data, you can either use provided datasets or create your own. Ensure your data follows the specified format so the model can process it effectively.

6. How to Run: Training

Once you’ve prepared your data, you can proceed with the training process:

• Collect and label your data.
• Adjust configurations in config/config.yaml.
• Run the training scripts in sequence.

7. Result and Performance

This project is intended for demonstration purposes and can work effectively if you use the same subjects consistently. To enhance performance, it’s essential to collect data from a broader demographic.

Troubleshooting Ideas

• If you find yourself facing errors, check if all dependencies are installed correctly.
• You may also refer to the tf-pose-estimation issues page for common problems.
• For adjusting features or including additional joints, modify the functions as suggested earlier.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.