Welcome to the exciting world of aiXcoder-7B! This powerful code large language model is designed for developers keen on exploring code generation and completion functionalities. Whether you’re a seasoned programmer or just starting your coding journey, aiXcoder-7B offers supportive tools to enhance your development process. This blog post will guide you through the essentials of setting up and utilizing the aiXcoder-7B model.

Model Introduction

As large code models evolve, aiXcoder has harnessed its potential to aid developers in practical scenarios. The aiXcoder 7B model has been meticulously trained on 1.2 trillion unique tokens and excels at code completion, comprehension, and generation. Notably, it outperforms similar models like Codellama 34B and StarCoder2 15B in key benchmarks.

What’s particularly fascinating is that this model is the foundation for future versions that will be fine-tuned for more specialized tasks such as test case generation and code debugging, which promises deeper support for programmers.

Quickstart

To get acquainted with aiXcoder-7B quickly, follow these steps:

Environment Requirements

Option 1: Build Environment

Ensure you have the following requirements installed:

• Python 3.8 or higher
• PyTorch 2.1.0 or higher
• sentencepiece 0.2.0 or higher
• transformers 4.34.1 or higher

Set up your environment using these commands:

conda create -n aixcoder-7b python=3.11
conda activate aixcoder-7b
git clone git@github.com:aixcoder-plugin/aiXcoder-7b.git
cd aiXcoder-7b
pip install -r requirements.txt

To improve performance, particularly for larger models, consider installing Flash Attention. Follow the steps below:

git clone git@github.com:Dao-AILab/flash-attention.git
cd flash-attention
MAX_JOBS=8 python setup.py install

Option 2: Docker

Using Docker can provide an isolated environment for running aiXcoder-7B. Here’s how to do it:

1. Install Docker on your machine.
2. Pull the Docker Image:

docker pull pytorch/pytorch:2.1.0-cuda11.8-cudnn8-devel

3. Run the Container:

docker run --gpus all -it -v /dev/shm:/dev/shm --name aix_instance pytorch/pytorch:2.1.0-cuda11.8-cudnn8-devel /bin/bash
pip install sentencepiece
git clone git@github.com:aixcoder-plugin/aiXcoder-7b.git
cd aiXcoder-7b

This method minimizes potential issues related to software versions and dependencies.

Model Weights

To access the model weights, visit: aiXcoder Base Download. The instruct model will be available soon!

Inference Example

Command Line Execution

Run the model inference directly via the command line:

torchrun --nproc_per_node 1 sess_megatron.py --model_dir path_to_model_weights_dir

Replace path_to_model_weights_dir with your actual model weights path.

Python Script Execution

You can also call the model through Python scripts, allowing greater flexibility:

from sess_megatron import TestInference
infer = TestInference()
res = infer.run_infer(
    code_string="",
    later_code="n",
    file_path="test.py",
    max_new_tokens=256,
)
print(res)

Troubleshooting

If you encounter issues during setup or execution, consider the following troubleshooting tips:

• Ensure you’ve installed all necessary libraries as listed in requirements.txt.
• Verify that you’re using compatible versions of Python and PyTorch.
• If using Docker, check that Docker has the required permissions for your machine’s GPU.

For further help and insights, feel free to connect with the vibrant community at fxis.ai.

Final Words

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.