Welcome to the delightful world of Facemoji, where your voice chatbot learns to mimic your facial expressions in a magical unison. This incredible Unity project is currently available for Android and consists of two remarkable modules: FaceTracking and AI. In this blog, I will guide you on how to set up and use Facemoji, ensuring that you can immerse yourself in this dynamic adventure!

Understanding the Components

Before diving in, let’s break down the two main modules of the Facemoji project to make it more relatable:

• FaceTracking: Imagine you have a puppet that mimics your every facial expression. Using OpenCV and Dlib, this module detects your facial movements, translates them into a Live2D model, and allows you to record animated gifs of your expressions.
• AI Chatbot: Picture a friendly companion who can chat with you using voice or text. This chatbot utilizes Turing Robot and Iflytek’s services to engage in conversations, tell jokes, forecast the weather, and much more. However, it currently speaks mostly in Chinese, so perhaps pick up a bit of that language for the best experience!

Setup Instructions

Let’s get you started with setting up Facemoji:

1. Download the essential components:
   • shape_predictor_68_face_landmarks.dat (Facial Landmark Detector)
   • Facemoji_Plugins_Assets_1.5.0.unitypackage (Streamlined OpenCV, Dlib, Live2D and Iflytek Assets Library)
2. Clone the GitHub repository by running the following command: git clone https://github.com/huihut/Facemoji.git
3. Create a new Unity project called FacemojiDemo.
4. Copy FacemojiAssets and FacemojiProjectSettings into your Unity project (FacemojiDemo).
5. Place shape_predictor_68_face_landmarks.dat into FacemojiDemo/Assets/StreamingAssets.
6. Import Facemoji_Plugins_Assets_1.5.0.unitypackage.
7. Select the Android platform.
8. Build and run!

Using Facemoji

FaceTracking

This module seamlessly tracks your facial movements. Try shaking your head, and watch how the character reflects your expressions—you’re the puppeteer here!

Recording GIFs

You can create short, fun gifs of your facial expressions. Here’s how:

• Locate the record button in the interface.
• Choose to record a 3-second GIF.
• The recorder states you might see include:
  • Recording: Ready to record
  • PreProcessing: Recording in progress
  • Paused: Compressing your GIF
• Your saved GIFs can be found in Application.dataPath at Android/storage/emulated/0/Android/data/com.huihut.facemoji/files.

Voice and Text Chat

Now, let’s dive into the chatbot features! You can engage with your AI buddy using voice or text. While the chatbot is great at numerous tasks such as chatting, telling stories, jokes, and more, remember that it currently only communicates in Chinese (but plans to speak English in the future!).

Troubleshooting Tips

If you encounter any hiccups during your Facemoji journey, consider the following troubleshooting steps:

• Ensure all necessary files are correctly placed in your Unity project.
• Verify that you have selected the Android platform correctly.
• Check the compatibility of any additional libraries you are using.
• Try re-importing packages if you face issues with functionality.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Happy coding and enjoy chatting with your newfound AI friend!