A curated (still actively updated) list of practical guide resources of LLMs. It’s based on our survey paper: Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond and efforts from @xinyadu. The survey is partially based on the second half of this Blog. We also build an evolutionary tree of modern Large Language Models (LLMs) to trace the development of language models in recent years, highlighting some of the most well-known models. These sources aim to help practitioners navigate the vast landscape of large language models (LLMs) and their applications in natural language processing (NLP). We also include their usage restrictions based on the model and data licensing information.

What You Will Find Here

• Latest news regarding key updates and added features
• A comprehensive list of practical guides
• Usage and restrictions of various models

Latest News

• We added a usage and restrictions section.
• We used PowerPoint to plot figures and released the source files.
• We corrected a logo for Tik.

Practical Guide for Models

Understanding large language models (LLMs) necessitates knowledge of how different types operate. Imagine trying to cook a fancy dish; each recipe (or model) has particular ingredients (data format) and cooking methods (architecture) that produce distinct outcomes.

• BERT-style Language Models:
  • BERT: A comprehensive study on language understanding.
  • RoBERTa: An optimized pre-training approach.
• GPT-style Language Models:
  • GPT-2: Exploring unsupervised learning tasks.
  • OPT: Open pre-trained transformer models.

Practical Guide for Data

Just as a great meal requires good ingredients, successful training of LLMs hinges on quality data. Think of pretraining data like the groceries for a cooking class – the better the selection, the more delightful the dish.

• Pretraining Data:
  • RedPajama
  • The Pile: An extensive dataset for language modeling.
• Test Data/User Data:
  • Shortcut learning survey
  • Robustness of ChatGPT

Troubleshooting

When exploring LLMs, you may encounter various challenges, such as incorrect model configurations or issues in data formatting. Here are some troubleshooting suggestions:

• Ensure that you use the correct data format compatible with your chosen model.
• Cross-verify model architecture and hyperparameters as per available documentation.
• In case of persistent errors, consider seeking help from the community forums or through pull requests.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.