๐๏ธ ๐ ๐
Flyte is an open-source orchestrator that facilitates building production-grade data and ML pipelines. Built with scalability and reproducibility in mind, it leverages Kubernetes as its underlying platform. With Flyte, user teams can construct pipelines using the Python SDK and seamlessly deploy them in both cloud and on-premises environments, enabling distributed processing and efficient resource utilization.
Build
Write code in Python or any other language and leverage a robust type engine.
Deploy & Scale
Either locally or on a remote cluster, execute your models with ease.
Get Started ยท Documentation ยท Resources
Table of contents
Quick start
- Install Flyte’s Python SDK
pip install flytekit
- Create a workflow (see example)
- Run it locally with:
pyflyte run hello_world.py hello_world_wf
**Ready to try a Flyte cluster?**
- Create a new sandbox cluster, running as a Docker container:
flytectl demo start
- Now execute your workflows on the cluster:
pyflyte run --remote hello_world.py hello_world_wf
Do you want to see more but don’t want to install anything? Head over to sandbox.union.ai. It allows you to experiment with Flyte’s capabilities from a hosted Jupyter notebook.
Ready to productionize? Go to the Deployment guide for instructions to install Flyte in different environments.
Tutorials
- Fine-tune Code Llama on the Flyte codebase
- Forecast sales with Horovod and Spark
- Nucleotide Sequence Querying with BLASTX
Features
Flyte offers various features that empower users in building and deploying robust machine learning workflows:
- Strongly typed interfaces to validate data.
- Code in any language using raw containers or choose a language SDK.
- Immutable executions for reproducibility.
- Data lineage tracking for workflows.
- Map tasks for parallel execution.
- Multi-tenancy support.
- Dynamic workflows that adapt as needed.
- Data visualization and monitoring.
- Cache task outputs to optimize performance.
- Notifications via Slack, PagerDuty, or email to stay updated on workflow changes.
Who’s using Flyte
Join the likes of LinkedIn, Spotify, Freenome, Pachama, Warner Bros., and many others in adopting Flyte for mission-critical use cases. For a full list of adopters and information on how to add your organization or project, please visit our ADOPTERS page.
How to stay involved
- Monthly community sync: Join the Flyte team for updates and progress sharing.
- Join the Flyte community on Slack for real-time communication and support.
- Subscribe to the Flyte Monthly newsletter.
- Tune into our YouTube channel for discussions and updates.
How to contribute
There are many ways to get involved in Flyte:
- Submit bugs and feature requests.
- Review the documentation and submit pull requests.
- Engage in discussions and share your experiences.
- Work on a help wanted or good-first-issue.
Troubleshooting
If you encounter any issues while working with Flyte, here are some troubleshooting ideas:
- Ensure your Flyte installation is correct.
- Check your network settings if you have trouble accessing clusters.
- Review the logs for errors in workflow executions.
- Reach out on Slack for community support.
For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.
At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.