Welcome to your guide on implementing BundleTrack, a cutting-edge framework for 6D pose tracking of novel objects without requiring instance or category-level 3D models. This article will walk you through the installation, setup, and execution of the BundleTrack code, along with troubleshooting tips to assist you.

Understanding BundleTrack

Imagine a skilled chef creating a new dish without an exact recipe; instead of relying on standard measures and techniques, they leverage their culinary skills to experiment with textures, flavors, and presentation. In a similar vein, BundleTrack operates without predefined 3D models for object tracking. Instead, it utilizes advanced techniques from deep learning for segmentation and robust feature extraction. The system maintains spatiotemporal consistency using memory-augmented pose graph optimization, allowing for accurate tracking under challenging conditions.

Getting Started: Environment Setup

Before diving into the code, you must prepare your environment. We recommend using Docker for ease.

Steps to Set Up the Environment

• Install Docker.
• Run the following commands to pull the required images:

docker pull wenbowen123/bundletrack:latest
docker pull wenbowen123/lf-net-release-env:latest

• Edit the dockerrun_container.sh file to update paths for BUNDLETRACK_DIR, NOCS_DIR, and YCBINEOAT_DIR.
• Execute:

bash dockerrun_container.sh

• Change directory to BundleTrack: cd [PATH_TO_BUNDLETRACK]
• Compile the project:

rm -rf build
mkdir build
cd build
cmake ..
make

Data Preparation

You’ll need to download several datasets and models to get started. Follow these steps:

• Download the weights for the feature detection network from the provided links and organize them correctly in the project directory.
• Similar steps apply to video segmentation network weights and precomputed masks — ensure you extract and place them in the specified directories.
• Download the NOCS dataset and ground-truth text pose files, ensuring your directory structure matches the required format.
• Repeat this process for the YCBInEOAT dataset.

Running Predictions

You’re almost there! You can now run predictions on your datasets:

For NOCS Dataset

• Open a new terminal:

bash lf-net-release/dockerrun_container.sh
cd [PATH_TO_BUNDLETRACK]/lf-net-release
python run_server.py

• Go back to the original terminal and run:

python scripts/run_nocs.py --nocs_dir [PATH_TO_NOCS] --scene_id 1 --port 5555 --model_name can_arizona_tea_norm

For YCBInEOAT Dataset

• Modify the config_ycbineoat.yml to specify the model name and directory.
• Open another terminal for server execution:

bash lf-net-release/dockerrun_container.sh
cd [PATH_TO_BUNDLETRACK]/lf-net-release
python run_server.py

• Run the YCBInEOAT prediction script:

python scripts/run_ycbineoat.py --data_dir [PATH_TO_YCBInEOAT] --port 5555 --model_name [The YCB object's name]

Troubleshooting

If you encounter issues, consider the following:

• Ensure all file paths and directory structures are configured correctly.
• Check that Docker is running properly and that the images were downloaded without errors.
• If the server doesn’t start, examine the log outputs in the designated folders for clues regarding potential errors.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

Following this guide will help you implement and run BundleTrack effectively for 6D pose tracking. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.