Deploying My First AI AGENT in Production!

as you can see from the title I have deployed my first AI agent so today I need your help in testing it let me explain why this video was sponsored by HubSpot okay so I guess I'm just going to blow this secret straight away and that is the clae tree hu model wow this is if you haven't tried it go ahead so you can see I put down some notes here so it has GPT 4 levish response if you look at the lat this llm Arena update you can see CLA hu now is in six place it's beating some older versions of gp4 that is just crazy and cloy Opus now has climbed to the top well deserved if you ask me but if you go back here you can see the CLA hik model has a 200k context window it's super fast and it's super cheap so only quarter of dollar per million input tokens so what I have been using it for is to create an AI agent that kind of leverages what I have called in context training to answer my YouTube comments so basically you can see here of course we have set a system message in our code so what I've been using here now instead of doing finetuning instead of doing yeah you can call it rag not exactly but I'm using what I have called in context training right so in the context we feed a bunch of instructions I have collected 40 plus example pairs so it's kind of a user comment from my YouTube channel and my answer so I'm going to show you all this how I set it up I also include the video video transcription from the video I'm going to use this on and of course we have to include the users YouTube comment in this context and we have some more instructions I'm going to show you all of this set up uh later and this means that we can kind of get the AI agent to answer in my style to the YouTube comments and that is what I need your help with today uh I need you to kind of go to this video just make any comment uh I'm going to show you an example here soon uh let me just bring that up straight away first I just want to say sorry to Daryl for being a part of the beta test of this even though you didn't know but he left a pretty interesting comment here about Invidia uh most recent GTC note and the bot responded uh even though it didn't have any information about the Nvidia Blackwell ships it kind of put together from the comment here uh that's a really interesting about the Nvidia blackw ships I have to check that out sounds like I could be a step towards I'm describing so with this inext training it kind of knows what the video is about and with information from the comment from Daryl here it can kind of bring up good answer back I think so it's pretty interesting to see uh again sorry about that uh so if you have time today and you want to leave a comment to help me test out this so here I kind of left a comment and I got a response and I just like yeah I rate this comment eight out of 10 right so if you have time to do that I would really appreciate it when it comes to integrating AI in the workplace writing great prompts for M Journey Chachi BT can be a game changer for efficiency but where do you start HubSpot the sponsor of today's video has got you covered with their free ebook AI trends for marketers this comprehensive guide is your key to unlocking the full potential of AI in your Marketing workflows in today's fastpaced landscape staying ahead of the curb is crucial for marketers by leveraging AI power tools and strategies you can streamline processes gain valuable insights and deliver personal experiences to your audience HubSpot ebook is packed with the latest trends and practical tips from understanding the basics of AI to exploring applications in social media email marketing and content creation you'll also discover how to choose the right AI tools how to integrate this into your existing marketing stack and measure their impact on your campaigns my favorite is a practical tip section HubSpot dives into the best practices for crafting effective prompts and enhancing the use of the tools you already use with real world examples that showcase how well-crafted prompts can lead to more accurate and valuable AI generated content and ultimately saving time and resources for your team so don't miss out on this opportunity and grab your free copy of the ebook by following the link in the description today a big thanks to HubSpot for sponsoring this video now let's go back to the project so what really makes this possible is that this model claw Tre hu is so cheap right and we can use the 200k token window to kind of fill in so much information and even though we can do that and when I run this now I have 8,247 around that is input tokens in every single response and output is going to be around 50 to 100 I think I did some calculations and the total price per comment is just 02153 around that doll per comment and that gives me like a $2.15 per th000 comments and that is a lot of comments so I would say that's really cheap right uh if you use GPT 4 or clae 3 for this uh that's going to cost you a lot and I think this is like 50 or like 100% cheaper than chat GT 3.5 Turbo with much better response and a much better context window so I would say uh CLA 3 Haiku has kind of enabled uh the AI agent dream I kind of had for some six to months to one year ago so I'm definitely going to be building out more of these now to try to help me with kind of my business right if you look at the workflow here it's a pretty simple setup so we have a YouTube comment so this is going to be fed via the YouTube API to our python Hub python code it's going to send a comment to CL hu with all the in context training we're going to answer that and that is going to be fed back to YouTube again and to respond to that single comment via the YouTube API so I set this system up to check for new comments every 3 minutes to kind of say F the quota so if you leave a comment you will get an answer back in at least three minutes right okay so I think we're just going to take a look at how I set this in context training up now and do a few tests and yeah let's do it so I'm not going to waste your time by going through every single line of the code here because this is a very specific use case for me running a YouTube channel right but you can mod this into every kind of use case you want if you have one but you if you're really interested in this specific use case just send me an email or something and we can work something out uh but let's see here so this is kind of my generate answer function so this is where the comments are going to come in so you can see I have a system message here and I have something a variable called in context training I know I say training here but training right and this is going to open a file I have named context. text uh but before we look at that let's take a look at the prompt here so we're going to use an F string we're going to say context is going to be equal to and here we are going to feed in our in context training right and then I just continue the prompt so B basically I just tried this learn from the context I referred back to that give a short but great answer in only lower case uh to the following YouTube comment and here again we kind of fetch the comment and feed that in here and I just end it with answer the YouTube comment in the style of Chris so the reason I can put in the style of Chris if we uh go and take a look at our in context training now if we start on top here you can see I have example comment so I just put that equal to and here I have put in 40 of these example pairs so maybe just a user put down all work great job keep it up and then I answer this so this is a comments I have answered so this is real world data right it's not synthetic so yeah you can see I put in yeah you can see it's 40 comments so we have these example pairs right so this is one pair uh I put down some extra information so I link to my YouTube membership uh should be in the description below my email right if someone asked for that should be. right okay uh some questions how to get access to GitHub that's something I always get send an email and to so I just try to feed as much info as possible in a transcript and here I kind of do the video transcripts uh I'm going to quickly show you how I do that if people are interested so this is going to be quite big so this is the full transcript of the video right so you can see it's quite long and you can see from the example comments info and the video transcript so I kind of refer back to all these uh variables here above your task is to mimic the way Chris would respond to comments also don't restate the user's question too much in your answer because that just takes up a lot of space right if I repeat the question every single time so this is kind of my context file so this is the on in context training file we also have the system esset so that's pretty simple so you're an expert in just I llms and machine learning you work for Chris from the YouTube channel all about AI your task is to mimic the way Chris would respond to comments so that is my simple but effective system message right and you can see we are using the CLA 3 hu model uh it says Max tokens uh 1024 but that is output tokens right so not input it doesn't really matter what you set here so yeah that is basically how I set this up and then we just going to run this so I think we're going to just do let's do one test and kind of see how fast we can get this back right okay so let's leave this comment here so yeah this is good uh what would you think would be the biggest challenge to create this system and how do I become a member to get access to the code so let's copy that let's go here and leave that comment and you can kind of see in the background now so this is every third minutes this is going to look for a new comment and if we get one it's going to try to answer it okay so you can see now we kind of picked up a new comment here so if we blow this up now a bit w that was big uh you can kind of see here uh so this is the answer it gave right if we scroll a bit up here sending prompt through an Tropic API and here is kind of the big prompt we sent so every single line here this was in The Prompt right and then we kind of got this response back if you zoom in a bit you can see it's 8,000 input tokens 45 output tanks the biggest challenge should probably with the compu and storage needed to save all the inner monologue thoughts become a member is easy just use the link in the the description to join up perfect that's a great answer right I think I'm very happy with that if we go back here to our video now you can see the comment is down here now so that worked great right and then I can leave this yeah I thought it was pretty good so I gave it an eighth out of 10 right okay perfect so be great if you can do that yeah that's basically how this works uh so if you kind of want to see how I transcribe uh to get the video transcript uh in this from my YouTube video I'm just going to quickly show you how I do that so I'm just using whisper and I just put my video into an MP3 format I just run this script here right and here you can kind of see it takes and transcribe everything into text so basically we can just comment this so basically our transcription pro video we can go to our context and just feed it in here so this adds a lot of information about the video so that our AI agent can answer questions about the video because it need to know the content right so for my experience so far it's a small test but maybe I know more today so I'm going to be feeding this video transcript you are watching now into this right uh and then we can see if he can answer some of the questions right so pretty cool so yeah that is what I wanted to share with you today so going forward trying to use CLA tree hu to create this small specific agent it's going to be something I will do and I will try to share as much uh information as I found out on this YouTube channel so again if you have time to leave a quick comment and maybe rate it if you want to do that you don't have to it it's up to you like you don't have to but I really appreciate it and yeah thank you for tuning in go check out CLA 3 Haiku it's awesome also check out Opus so yeah I'm kind of happy we have something that kind of Beats gp4 it's about time right uh but other than that thank you for tuning in have a great weekend Easter call it what you want and I'll see you again on Sunday

Channel: All About AI

Views: 10,309

Keywords: ai agent, ai agents, claude 3, claude 3 haiku, ai automation, in context training, autonomous ai agents, youtube ai agent

Id: xrNFJdedysU

Length: 12min 22sec (742 seconds)

Published: Wed Mar 27 2024