Online Demo
GitHub
Paper
Discord
Table of Contents
Usage
To use the OpenChat model effectively, we suggest installing the OpenChat package by adhering to the installation guide in our repository. Start the OpenChat OpenAI-compatible API server using the command provided in the table below. This server optimizes for high-throughput deployment using vLLM and operates on consumer GPUs with 24GB RAM.
To enable tensor parallelism, append --tensor-parallel-size N to the serving command. Once the server is up, it listens on localhost:18888 for requests, complying with the OpenAI ChatCompletion API specifications. Please see the example request below for guidance:
python -m ochat.serving.openai_api_server \
--model openchat/openchat-3.5-0106 \
--engine-use-ray \
--worker-use-ray detailsBenchmarks
Our model’s benchmark performance is rated as follows:
Model # Params Average MT-Bench HumanEval BBH MC AGIEval TruthfulQA MMLU GSM8K BBH CoT
------------------------------------------------------------------------------------------------------------------------------
**OpenChat-3.5-0106** **7B** **64.5** 7.8 **71.3** **51.5** **49.1** 61.0 65.8 **77.4** 62.2
For a better understanding, imagine these benchmarks as traits of an athlete evaluating their performance in various competitions. The scores reflect the athlete’s ability, endurance, and agility in completing the tasks at hand.
Limitations
While OpenChat shows exceptional capabilities, it does carry inherent limitations:
- Complex Reasoning: It may struggle with sophisticated reasoning tasks.
- Hallucination: Users should be cautious, as the model may produce inaccuracies.
- Safety Issues: Some outputs might be harmful or biased, necessitating careful supervision.
License
Our OpenChat 3.5 code and models are distributed under the Apache License 2.0.
Citation
If you wish to reference our work in academic settings, please use the following citation format:
@article{wang2023openchat,
title={OpenChat: Advancing Open-source Language Models with Mixed-Quality Data},
author={Wang, Guan and Cheng, Sijie and Zhan, Xianyuan and Li, Xiangang and Song, Sen and Liu, Yang},
journal={arXiv preprint arXiv:2309.11235},
year={2023}
}💌 Contact
We look forward to your feedback and collaboration as we advance this engaging project. Our team is always interested in hearing your insights and ideas!
Troubleshooting
If you encounter any issues while using OpenChat, consider these common troubleshooting tips:
- Ensure your setup meets the hardware requirements for optimal performance.
- Review the serving command in case any flags are incorrectly set.
- Verify that the API keys used are allowed and correctly configured.
- Check the logs (if enabled) for detailed error messages that can provide clues.
For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.
At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.