Created by Philipp Lindenberger · Paul-Edouard Sarlina · Marc Pollefeys
ICCV 2023
Paper • Colab • Poster • Train your own!
What is LightGlue?
LightGlue is a deep neural network designed for matching sparse local features across image pairs. With an adaptive mechanism, LightGlue accelerates the matching process for simple pairs while skillfully managing computational complexity for more challenging ones.
Installation and Demo
You can install LightGlue using the following commands:
git clone https://github.com/cvg/LightGlue.git
cd LightGlue
python -m pip install -e .To see it in action, we provide a demo notebook that showcases feature extraction and matching between an image pair. You’ll find that using LightGlue is as easy as pie!
Basic Usage
Imagine a librarian trying to match books across two different shelves. Each book is unique, and the librarian knows the title, author, and year of publication. The librarian employs a methodical approach. Similarly, LightGlue matches images through various steps:
- Extract features from both images (like titles from books).
- Match these features (as the librarian would pair similar books).
- Return the indices of matching points (the results of a job well done).
Example Code
from lightglue import LightGlue, SuperPoint, load_image
# Load feature extractor
extractor = SuperPoint(max_num_keypoints=2048).eval().cuda()
matcher = LightGlue(features=extractor).eval().cuda()
image0 = load_image('path/to/image_0.jpg').cuda()
image1 = load_image('path/to/image_1.jpg').cuda()
feats0 = extractor.extract(image0)
feats1 = extractor.extract(image1)
matches = matcher(image0=feats0, image1=feats1)Advanced Configuration
LightGlue offers a level of customization, enabling you to tailor it to your needs:
- n_layers: Adjust the number of layers for faster inference.
- flash: Enable FlashAttention to improve speed.
- depth_confidence & width_confidence: Fine-tune iterative process parameters.
- filter_threshold: Manage the confidence in matches.
Performance Insights
LightGlue is engineered for speed. On a GPU (RTX 3080), it can achieve up to 150 FPS with 1024 keypoints. This translates to a significant performance boost—4 to 10 times faster than prior solutions like SuperGlue.
For CPU evaluations (Intel i7 10700K), it still performs at 20 FPS with 512 keypoints.
Troubleshooting
If you face issues during installation or runtime, consider the following steps:
- Ensure you have the required dependencies installed.
- Check your GPU memory; running out of memory can cause crashes.
- Verify that your input images are correctly formatted.
- Adjust the parameters according to your hardware as specified in the configuration section.
- If you still encounter issues, consult the community or documentation for further insights.
For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.
Conclusion
At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.