I love these AI audio style transfers by “there-I-ruined-it”. So I did some digging. He posts quite a bit on his IG account. Enough to show him doing his own vocals. My educated guess on his workflow & tools: 1) get or make underlying music track. Could also STEM an existing track with open source tools (DEMUCS) or paid service like Fadr. com 2) perform vocal lyrics over track using his voice with lyrics from another song, to the original melody. 3) take his vocal track into a style transfer software. Likely “so-vits-svc”. 3b) create a cloned voice of the original artist if it doesn’t exist. But there are hundreds of artists already cloned. 3c) ElevenLabs speech-to-speech might also work but he was doing this long before that was released. & that would violate EL’s terms. 4) overlay the cloned voice / style-transferred version on the track. The key to this being so on point is his ability to nail the pre-cloned vocals with his own voice. Which takes quite a bit of talent. 🔥
I love these AI audio style transfers by “there-I-ruined-it”. So I did some digging. He posts quite a bit on his IG account. Enough to show him doing his own vocals. My educated guess on his workflow & tools: 1) get or make underlying music track. Could also STEM an existing track with open source tools (DEMUCS) or paid service like Fadr. com 2) perform vocal lyrics over track using his voice with lyrics from another song, to the original melody. 3) take his vocal track into a style transfer software. Likely “so-vits-svc”. 3b) create a cloned voice of the original artist if it doesn’t exist. But there are hundreds of artists already cloned. 3c) ElevenLabs speech-to-speech might also work but he was doing this long before that was released. & that would violate EL’s terms. 4) overlay the cloned voice / style-transferred version on the track. The key to this being so on point is his ability to nail the pre-cloned vocals with his own voice. Which takes quite a bit of talent. 🔥
@blizaine This seems pretty spot on. Also if someone had an AI that could do this in one click (or very little effort/knowhow) they'd be sitting on a $50-100mil product and prob not just using it to get views on sm.
@blizaine I love these as well! They are amazing, AI or not….just to put something like this together is amazing. But your workflow above looks accurate…I wonder how as well.
@blizaine Some of these are hilarious! Some also kind of mess with my head 👀😂