How to Use Hera for Orchestrating Python Code on Argo Workflows

Aug 3, 2022 | Data Science

Welcome to your guide on leveraging Hera, a powerful tool that simplifies orchestration of Python code on Argo Workflows. Hera allows you to construct and submit workflows entirely in Python, bringing you closer to a seamless experience in your data operations. Whether you are a beginner or a seasoned programmer, this article will help you understand how to get started with Hera.

Getting Started with Hera

To embark on your Hera journey, follow the quick start guide linked here. You’ll learn how to deploy and configure Hera in your environment, setting it up for orchestrating your workflows easily. But first, let’s delve into some basic concepts and examples.

Understanding the Basics of Hera

Basic Code Structure

Let’s use an analogy to describe the basic structure of the code within Hera. Think of constructing a movie script where different characters (steps) interact to create a story (workflow). Each character has their own lines to deliver and can perform their actions simultaneously or in sequence, similar to how steps within a workflow function.

from hera.workflows import Steps, Workflow, script

@script()
def echo(message: str):
    print(message)

with Workflow(generate_name="single-script-", entrypoint="steps") as w:
    with Steps(name="steps") as s:
        echo(name="A", arguments={"message": "I'm a step"})
        with s.parallel():
            echo(name="B", arguments={"message": "We're steps"})
            echo(name="C", arguments={"message": "in parallel!"})
        echo(name="D", arguments={"message": "I'm another step!"})

w.create()

In this script, we define a simple workflow with four steps (characters) that interact in a sequence. Steps B and C perform their tasks in parallel, illustrating how Hera can execute multiple actions simultaneously, just as you would see multiple scenes happening at once in a movie.

Creating Directed Acyclic Graphs (DAGs)

Another Perspective on Workflow Design

DAGs in Hera allow for structured workflows that follow defined dependencies, akin to a story where certain events must occur before others unfold. In this case, the script below demonstrates how to set up a simple DAG using Hera:

from hera.workflows import DAG, Workflow, script

@script()
def echo(message: str):
    print(message)

with Workflow(generate_name="dag-diamond-", entrypoint="diamond") as w:
    with DAG(name="diamond"):
        A = echo(name="A", arguments={"message": "A"})
        B = echo(name="B", arguments={"message": "B"})
        C = echo(name="C", arguments={"message": "C"})
        D = echo(name="D", arguments={"message": "D"})
        A >> [B, C] >> D

w.create()

This script depicts a simple storytelling scenario where event A must occur before both B and C can happen, which in turn leads to a conclusion, D. This logic mimics the real-world dependencies often encountered in project management.

Installation of Hera

To get started, you must install Hera. You have a couple of options for installation:

  • PyPI: Run pip install hera
  • GitHub Repo: Use python -m pip install git+https://github.com/argoproj-labs/hera --ignore-installed

Be aware that Hera has undergone a name change from hera-workflows to hera, with versions published for backward compatibility.

Troubleshooting Steps

If you encounter issues during installation or while running your workflows, consider the following troubleshooting ideas:

  • Ensure that your Kubernetes cluster is up and running, and that the Argo server is properly configured.
  • Refer to the Argo Workflows quick start guide for deployment instructions.
  • If you’re facing authentication problems, verify your Bearer token setup or consider port forwarding your Argo server to localhost:2746.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

With Hera’s ease of use, you can orchestrate complex workflows in Python without feeling overwhelmed. By simulating workflows as stories or movies, you can manage processes in a user-friendly manner. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Explore Further

Don’t forget to check out more examples and use cases directly in the Hera examples documentation for inspiration on how to elevate your workflow management strategies!

Stay Informed with the Newest F(x) Insights and Blogs

Tech News and Blog Highlights, Straight to Your Inbox