This is an old revision of the document!
AI Cloning
By AI cloning in this context we mean creating an avatar that can animate and talk to represent a living person with artificial intelligence (AI).
The Background Story
Since 2024, I've been working on this AI-powered cloning project. It started as a way to challenge myself and learn the ropes. It quickly grew into a massive, multi-year project, with the ultimate goal of duplicating myself for my daughter and wife after my death (you never know when my time will come), and ultimately living forever using future technologies like mind uploading.
Requirements for Creating an AI Clone
While creating an AI clone sounds complicated, but if you think in multiple blocks for each function and work them out. It seems less difficult.
* For talking I use a Text To Speech (TTS) engine Chatterbox TTS, which is capable of voice cloning;
* For listening I use a Speech To Text (STT) system Whisper-FastAPI;
* For creating a 3D model of myself Avaturn;
* For teaching my model to animate I want to use MediaPipe Holisitc;
* For processing text and reply I am using Ollama with the Meta Llama 3 Model, this model is great at roleplay;
* For the 3D engine I use pure WebGL instead of a game engine like Unity 3D or Unreal Engine which seems a more choice.