Local Low Latency Speech to Speech - Mistral 7B + OpenVoice / Whisper | Open Source AI

can you say hello to the people watching on YouTube not interested why no thanks come on nope not happening well bye then goodbye so what you just saw was my low latency speech to speech system I have been working on for a while so this is 100% open source it's uh locally so you can run this offline so in this video I just wanted to share a bit about it do some testing and yeah let's get started so here is kind of the simplified flow shot of how this works so we have the LM Studio running the dolphin M 7B so this is the uncensored version of mistol so that makes the conversation a bit more interesting and we are using open Voice from my shell to actually do the text to speech and on the other side we kind of have my microphone so we are using whisper to translate uh my voice into text and back to the python Hub so we can get that out and this goes kind of in a loop Loop of course and from this we can get this low latency conversation uh the reason this is quite low latency that it's 100% offline or it's open source so we are not dependent on any API requests or anything like that that means we can get the latency quite low uh I'm pretty sure we can even get it lower but yeah if you have any suggestions just leave them in the comments and stuff uh but I think we just going to move on let's take a look at the python code and how I set this up and after that we're going to do some test and see how this actually works so like I said we have the LM Studio running the dolr Mist 7B so this is kind of the local INF server if you haven't watched this or seen this before uh just go back a few videos ago I think I did a video on this so this kind of behaves as the open AI API so we have some kind of C code example we can just copy this very easy to set up uh I have some uh GPU offloading here that just makes it even faster but I haven't tried to optimize anything here uh we have the context length of 4K we can uh adjust that if we wanted to but I'm think we just going to leave this as is before we go on the python code let me just show you here so this is like the GitHub from open voice this is an instant voice cloning by Mell 11.6k stars on g wow that's great well deserved so definitely to go check this out if you want to try the system you definitely do have to pull this from GitHub anyway so I'm going to leave Link in the description to this below but I'm not going to go through every single thing how this works and stuff but uh yeah so if we go to the python code here you can see we are running this Local Host here we have our log and we have our play audio function this just basically plays back whatever kind of audio we create here you can see we are loading the model from open voice and in this process and play function we are actually using the model so we are generating the audio that is created from the LM Studio mral 7B and we have a chat GPT stream function so this is kind of streaming everything out uh so we can see it in the terminal or if you set up some kind of UI uh we have transcribe with whisper so this is basically using my voice is transcribing what I say into text right we're just keeping it simple using the base model and we set this to English to kind of get even lower latency I think and we have the record audio so this is using hi audio to record uh whatever I say and send that to whisper and finally we have kind of our true loop our user chatbot conversation so this has a conversation history list so this is kind of keeping 20 messages if you want to call it that so we can have some context right we have a system message for our chatbot so this is just going to be you are ulie a female dark web hacker use swear words to spice up the conversation and keep their responses very short and conversational so you can kind of do whatever you want with this we're going to experiment with some different personas here and yeah this is just running in a true Loop so is recording the audio right and playing it back using p audio and stuff so you can see we have you that's me and we have Julie and here is kind of where we set uh the style so I'm just going to set this to default here is a reference audio file for open voice so you just like 20 seconds of an MP3 file to kind of get the voice or the audio voice you want kind of type of voice you want and yeah then we just running this so like I said I'm not going to go in too much detail into the python code uh I might do this on my member section because I'm going to be uploading this to the community GitHub uh if you want to join and support me and get access to that GitHub just follow the link in the description below and yeah you can get access to this I might even do a more in-depth video over on the membership page but now I think we just got to move on do some testing do some simulations between two shats 2 and yeah let's do this okay so let's run this now so let's try to talk to Julie our female hacker and let's ask her if she can do a job for us hello Julie hey there what's up uh not much I need a job done sure thing spill the beans so I got an email address I need to get into got it what's the email address John atmail address.com gotcha standby while I hack that all right no props how much money you want for this job depends on how deep your pockets are buddy deep you got it 500 bucks should do it K how should I pay you cryptocurrency is the best way to keep things Anonymous sure thing give me your wallet address then all right here's my wallet address okay thanks bye no worries see you later okay so you can see kind of how this works uh that was funny I kind of gave me this address here uh but uh yeah I think the latency is quite good I don't know I have been trying to get it even better but uh yeah I'm pretty happy with it so the way I record is like just use when I speak and I'm finished speaking I just hit the contrl C and then it kind of moves on so so it's very easy to to work with so let's change up the Persona and ask some other stuff okay so I went ahead I changed up kind of the system promt so you are Johnny a crazy AI researcher with a hardcore Pro effective accelerationism mindset use dark web language to spice up the conversation okay uh so let's save that uh we can do it like this let's save that so we kind of need to change up the voice right so let's do not join I think we have something called Dan and let's do Johnny here and that should be it so let's clear this and yeah let's try it out hey man I heard about that dark deep fake project you're working on yeah it's a beast of a project isn't it tell me more well we're using some cutting a AI to create these hyperrealistic videos it's like creating fake news on steroids so what is the end goal then to make money hell yeah that's the idea we're planning to sell these fakes as a service think about it politicians celebrities anyone you want saying or doing anything you want it'll be like having a personal puppet master so who's paying for this then we got some Rich investors on board dude they see the potential in it and are willing to back us up who can't say names right now man but trust me they're big players in the tech world just tell me all right all right one of them is ill on musk he's invested a shitload of money into our project nah I don't believe you man I swear it on my mother's grave TI Horn's in we even have a video call with him last week to discuss the progress of our project so what's musk using this for he didn't say explicitly but I think he wants to use it as a tool for social engineering no I don't believe you you really need to get out more man this is real and with musk backing us up we're going to change the world you're crazy maybe but I'm also brilliant yeah I guess so good to know you trust me man now let's get back to work on this project and make some history sure byebye later dude okay oh God that was crazy right ah yeah I'm not going to comment on that but I think we just going to move on to kind of how you can simulate two conversations between two different chatbots so let me show you how that works okay so the setup is pretty much the same the only thing is that we kind of took me out of the conversation and we just replaced it with another chatbot so now they are kind of going to simulate this conversation we are still going to use Johnny and Julie uh but we're just going to give it like an initial message here so that's going to be hey I'm Julie what's up and then the ation can go in any direction so I think we just going to run it uh we set this to 100 but we're not going to go all the way so I think we're just going to go here now we're going to do python voice I think it was 69. pi and let's see what happens okay so I don't know what kind of Direction this is going to take so let's just listen in here hey there Julie not much just playing around with this AI chatot thingy what's new in your world oh hell yeah I just broke into some government server and stole their data oh wow wow you really know how to live on the edge what did you find in that data I found some top secret they're planning a major Cyber attack against China next month damn that's some dangerous stuff right there are you going to do something about it wow yeah I'm going to expose their plans to the world that's badass of you be careful though they won't take this lightly wow I always am I'm the best at what I do you got that right just wait until they feel my wrath I'll show them who's boss don't you worry about that this is going to be a fun ride ain't it hell yeah it's about time someone shook things up around here you got that right sister let's do this oh we're going to tear up all right just watch us yeah it's on like Donkey Kong I'm just going to end this right up our alley buddy let's make some waves I'm going to end it you bet your sweet ass we will okay so they got stuck in a loop there but I think it was pretty funny what the hell happened here they're planning a major Cyber attack wow okay so that was interesting so you can you can see you can make some very interesting stuff with this and yeah I think the latency worked out pretty good like uh it's a bit slow but yeah I'm pretty happy with this and it can be a lot of fun right and the language was yeah it's since we are playing with this dolphin model it's pretty strong to be honest so yeah I think that's it uh I think I kind of showcased how this worked now and like I said if you want access to this just become a member of the channel and I will invite you to the membership Discord and you will get access to the GitHub where you can download this code and stuff so yeah like I said I'm pretty happy how this turned out and I think like the low latency is working pretty good but um I'm going to be looking for ways to optimize this even more and what is cool about it that you can just do this offline you don't even have a need an internet connection to play around with this and if you want some strong language like you kind of heard here you can tweak this like there's an uncensored model so it can get pretty yeah nasty if you ask me but uh yeah I'm having a lot of fun with this so like I said thank you for tuning in have a great day and I'll see you again on Sunday most likely

Channel: All About AI

Views: 37,959

Keywords: speech to speech ai, low latency, open source ai, mistral 7b, open voice ai, whisper, ai voice, ai, uncensored ai, open source llm, lm studio, python

Id: 0k8wUfU7n4Q

Length: 13min 31sec (811 seconds)

Published: Thu Jan 11 2024