So I turned my VOICE into an anime WAIFU using ChatGPT and AI...

Introduction

In an exciting journey of combining technology and creativity, I endeavored to create an AI-driven chatbot that would interact with players in a popular game, Apex Legends. My primary objective was to give my bot an anime waifu persona, bridging the gap between gaming and entertaining voice interactions. Here’s how I achieved this feat using ChatGPT and various other AI technologies.

Building the AI Chatbot

Initially, I employed ChatGPT to help me conceive the idea and create the foundational code for an AI chatbot inspired by characters from anime. The challenge was to enable the chatbot to understand what's happening in Apex Legends while also giving it a compelling personality. However, the bot was still in its infancy during this phase.

Experimenting with Text-to-Speech

During my research, I stumbled upon some intriguing applications that could elevate the functionality of my AI chatbot. One of the most crucial implementations was text-to-speech (TTS). Many TTS engines available were either monotonous or locked behind paywalls. My quest for a free yet dynamic solution led me to discover a text-to-speech program called VoiceVox, a deep learning synthesizer developed in Japan that boasts an array of emotional voice options.

The challenge was that the documentation was primarily in Japanese, posing a significant barrier. Fortunately, I relied on the assistance of friends, including an expert in machine learning and another skilled in reverse engineering, to unlock the mysteries of the software.

With their help, I was able to create a function that took Japanese text as input and output an anime waifu's voice. This was a significant step towards bringing my vision to life.

Introducing Speech Recognition with Whisper AI

Next, I wanted to introduce speech recognition capabilities using OpenAI's Whisper AI. This technology is renowned for its robust performance, even when faced with thick accents and multilingual inputs. Similar to VoiceVox, Whisper AI also has a Docker image, which allowed me to run it seamlessly alongside my other applications.

I implemented functionality into my project that recorded audio from my microphone when holding down a specific key. This audio file would be processed by Whisper AI, which converts words from speech into text. Following this transformation, I had to tackle the issue of language translation since VoiceVox only understood Japanese input while Whisper AI only provided English output.

Bridging the Language Gap with DeepL

Realizing that we needed a middleman, I sought an AI translator and ended up with DeepL, which is known for its natural and conversational translation capabilities. After setting up my API key, I created an end-to-end solution where:

1. I could speak into my mic while holding down a key.
2. The Python application would record and save the audio file.
3. Whisper AI would process the audio to generate English text.
4. This text would then be translated into Japanese using DeepL.
5. Finally, the Japanese text would be sent to VoiceVox to synthesize the anime waifu voice.

Testing the System

After laying a solid foundation, it was time for a test run. Despite some initial lag due to the heavy resource usage of my Python program or possible server delays, I was able to produce real-time audio output in an anime voice based on my spoken words. The functionality to communicate in Japanese through an anime persona was thrilling.

While the system wasn't perfect and had a few glitches (like humorously incorrect translations), it effectively showed promise. The overall experience highlighted the potential of technology in overcoming language barriers and enriching the gaming experience.

The initial intention was to create a simple side project, but it ended up evolving into a revolutionary tool capable of facilitating conversation across languages, particularly in gaming environments such as Apex Legends.

If you're intrigued by the possibilities of using AI for creative projects like this, let me know, and I might just continue and reveal further updates!

Keyword

• AI
• Chatbot
• Apex Legends
• Anime Waifu
• Text-to-Speech
• Whisper AI
• DeepL
• Speech Recognition
• Language Translation
• Docker

FAQ

Q1: What technologies did you use for the project?
A1: I used ChatGPT for coding assistance, VoiceVox for text-to-speech, Whisper AI for speech recognition, and DeepL for language translation.

Q2: What was the main goal of the project?
A2: The main goal was to create an AI chatbot that could interact intelligently in Apex Legends with a personality reminiscent of an anime waifu.

Q3: How did you handle the language barrier in the project?
A3: I utilized DeepL for translating English text generated by Whisper AI into Japanese, which was then fed into VoiceVox for voice synthesis.

Q4: Did the system work perfectly during testing?
A4: The system showcased promising functionality, but there were some glitches and lag issues that were encountered during testing.

Q5: Can others replicate this project?
A5: Yes, I plan to open-source the code so others can use it for similar applications in gaming and beyond.