Build Time Aware Visualizations of Multimodal Data
Use the Rerun SDK (available for C++, Python, and Rust) to log data like images, tensors, point clouds, and text. Logs are streamed to the Rerun Viewer for live visualization or to file for later use.
A Short Taste
Here’s a simple snippet to get you started with Rerun:
import rerun as rr # pip install rerun-sdk
rr.init("rerun_example_app")
rr.connect() # Connect to a remote viewer
# rr.spawn() # Spawn a child process with a viewer and connect
# rr.save(recording.rrd) # Stream all logs to disk
# Associate subsequent data with 42 on the “frame” timeline
rr.set_time_sequence(frame, 42)
# Log colored 3D points to the entity at path "points"
rr.log("points", rr.Points3D(positions, colors=colors))
This sample code snippet is like setting up a concert, where:
- The
rr.init()line represents the stage being set up (your application). rr.connect()acts like the musicians tuning their instruments to connect with their audience (the viewer).- When you log data, it’s similar to a live performance where each musician plays their unique piece, which everyone gets to enjoy and experience.
Getting Started
To dive into Rerun, start with the following links for your programming language:
- C++ Installation
- Python Installation: pip install rerun-sdk or on conda
- Rust Installation: cargo add rerun
Installing the Rerun Viewer Binary
To stream log data over the network or load .rrd data files, you need the Rerun binary. This can be installed using:
pip install rerun-sdk
# or for Rust users
cargo install rerun-cli --locked
Note that only the Python SDK comes bundled with the Viewer, while C++ and Rust rely on separate installations.
Documentation
Explore the Rerun documentation:
Status
Rerun is in active development. Expect breaking changes! Note that:
- Multi-million point clouds may be slow.
- The viewer may slow down with too many entities.
- The data to visualize must fit in RAM. For methods to limit memory use, check this link.
What is Rerun For?
Rerun aids in understanding complex processes involving rich multimodal data. It’s designed for industries like robotics, simulation, and computer vision, among others. Here’s a sample use case: suppose you’re building a vacuum-cleaning robot that bumps into walls.
Example Use Case
Debugging this requires more than text logs. You need a tool to visualize the robot’s understanding of its world—like the RGB camera feed, depth images, and 3D maps. With Rerun, you can identify issues like glare affecting sensor performance or other bugs that obscure accurate object detection.
As you visualize the process, you might brainstorm ideas for algorithm improvements or new test cases. Rerun not only helps to debug but also to enhance your project’s capabilities.
Business Model
Rerun operates on an open-core model, ensuring that all components remain free and open-source, targeting individual developers while planning a commercial product for team-centric computer vision and robotics solutions.
Troubleshooting
If you encounter issues, consult the troubleshooting section of the documentation linked above. For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.
At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.