Automatic Agent Learning from Scratch for QA via Self-Planning

Learn about the innovative AutoAct framework that simplifies automated agent learning by synthesizing planning trajectories.

Table of Contents

• Acknowledgement
• Overview
• Installation
• Self-Instruct
• Self-Planning
• Citation

Acknowledgement

Our code of training module is referenced and adapted from FastChat, while the code of inference module is implemented based on BOLAA. We also used various baseline codes including ReAct, Reflexion, BOLAA, Chameleon, ReWOO, FireAct. We leverage LangChain with open models via FastChat. Thanks for their great contributions!

Overview

The realm of language agents has witnessed significant advancements, yet many current systems continue to grapple with excessive reliance on costly and non-reproducible data. In light of this, we proudly present AutoAct: an innovative automatic agent learning framework that operates without the need for large-scale annotated data. The framework employs an ingenious self-planning technique to automatically synthesize planning trajectories from limited data and a tool library. By doing so, it addresses the challenge of compelling a single model to manage multiple functions effectively.

Picture a team of workers at a factory. They each have different skills: some excel at assembling parts, while others are great at quality control. AutoAct functions like a meticulous factory manager who analyzes the tasks ahead and assigns roles based on each worker’s specialty. Instead of requiring extensive training data, it synthesizes its own workflows – or trajectories – to optimize results!

Installation

To get started with AutoAct, follow these commands:

bash
git clone https://github.com/zjunlp/AutoAct
cd AutoAct
pip install -r requirements.txt

Self-Instruct

In order to generate a sufficient amount of task data, conduct self-instruct on the Meta-Agent using the following command:

bash
python Self_Instruct/data_generation.py \
    --source_data Self_Instruct/Meta_sample/Meta_Hotpotqa.json \
    --target_data Self_Instruct/hotpotqa_metaqa.json \
    --dataset_name hotpotqa \
    --generate_all_num 800 \
    --generate_per_round_num 10 \
    --model_name llama-2-13b-chat

The above command generates a set of instructional data, ensuring the parameters are optimized for efficiency.

Self-Planning

Automatic Tool Selection

To automate the selection of applicable tools for tasks, execute:

bash
python Self_Planning/Tool_Selection/tool_selected.py \
    --model_name llama-2-13b-chat \
    --task_name ScienceQA \
    --top_k 40 \
    --top_p 0.75 \
    --max_tokens 1024 \
    --tool_save_path Self_Planning/Tool_Selection/task_name_Tools.json

Information on selected tools will be saved to the designated path.

Trajectories Synthesis

For synthesizing high-quality trajectories, use:

bash
python Self_PlanTraj_Syn/run_task.py \
    --agent_name ZeroshotThink_HotPotQA_run_Agent \
    --llm_name llama-2-13b-chat \
    --max_context_len 4096 \
    --task Hotpotqa \
    --task_path Self_Instruct/hotpotqa_metaqa.json \
    --save_path Self_PlanTraj_Syn/output/hotpotqa_train_data.jsonl

This command collects the best trajectories as training sources for self-differentiation.

Self-Differentiation

To distinguish roles among various sub-agents, run:

bash
for agent in plan tool reflect; do
    echo "####################"
    echo $agent
    echo "####################"
    CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 deepspeed Self_PlanTrain/train_lora.py \
        --model_name_or_path llama-2-13b-chat \
        --lora_r 8 \
        --lora_alpha 16 \
        --lora_dropout 0.05 \
        --data_path Self_PlanTraj_Syn/output/data_$agent.json \
        --output_dir Self_PlanTrain/lora/HatpotQA13b-$agent-5-epoch \
        --num_train_epochs 5 \
        --per_device_train_batch_size 2 \
        --per_device_eval_batch_size 1 \
        --gradient_accumulation_steps 1 \
        --evaluation_strategy no \
        --save_strategy steps \
        --save_steps 10000 \
        --save_total_limit 1 \
        --learning_rate 1e-4 \
        --weight_decay 0. \
        --warmup_ratio 0.03 \
        --lr_scheduler_type cosine \
        --logging_steps 1 \
        --fp16 True \
        --model_max_length 4096 \
        --gradient_checkpointing True \
        --q_lora False \
        --deepspeed Self_PlanTrain/deepspeed_config_s3.json \
        --resume_from_checkpoint False
done

This empowers each sub-agent with specialized functionalities, ensuring efficient task management.

Group Planning

To process new tasks through group planning among sub-agents:

bash
python Self_Planning/Group_Planning/run_eval.py \
    --agent_name ZeroshotThink_HotPotQA_run_Agent \
    --plan_agent plan \
    --tool_agent tool \
    --reflect_agent reflect \
    --max_context_len 4096 \
    --task HotpotQA \
    --task_path Self_Planning/Group_Planning/benchmark_run/data/hotpotqa \
    --save_path Self_Planning/Group_Planning/output/13b

Thus the team works together efficiently to achieve the desired outcomes.

Citation

If you leverage AutoAct in your work, please cite our repository using the following:

bibtex
@article{DBLP:journals/corr/abs-2401-05268,
  author       = Shuofei Qiao and Ningyu Zhang and Runnan Fang and Yujie Luo and Wangchunshu Zhou and Yuchen Eleanor Jiang and Chengfei Lv and Huajun Chen,
  title        = AUTOACT: Automatic Agent Learning from Scratch via Self-Planning,
  journal      = CoRR,
  volume       = abs/2401.05268,
  year         = 2024,
  url          = https://doi.org/10.48550/arXiv.2401.05268,
  doi          = 10.48550/ARXIV.2401.05268,
  eprinttype    = arXiv,
  eprint       = 2401.05268,
  timestamp    = Thu, 25 Jan 2024 15:41:08 +0100,
  biburl       = https://dblp.org/rec/journals/corr/abs-2401-05268.bib,
  bibsource    = dblp computer science bibliography, https://dblp.org
}

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Troubleshooting Section

If you encounter issues during the installation or execution of commands, here are some common troubleshooting ideas:

• Dependency Errors: Ensure all required packages are up to date. Running pip install --upgrade -r requirements.txt may resolve issues.
• CUDA Errors: Check if your system has compatible CUDA and cuDNN versions installed.
• Data File Not Found: Verify that all paths provided in the commands are correct and files exist.
• Resource Limitations: If you are running out of memory, consider lowering the batch size in your training commands.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.