LangSmith For Beginners | Must know LLM Evaluation Platform 🔥

hello guys welcome back in this video let's go through Lang ismith from Lang chain I had created four different videos about Lang Smith and also the Lang Chen hob which is inside the Lang Smith 5 6 months before as you can see on the screen when it was introduced let's say but it was not generally available you have to be in the waiting list and once they approved you you can use it many of you find that okay I I can't use it kind of questions what they did now in February 15 that now the Lang Smith is generally available and there is no waiting list also they have added new functionality into the Lang Smith website why not create a new video out of it so that you get the fresh uh fresh look of Lang Smith how the video will proceed is the steps as it is mentioned here I'll will first show you what is Lang Smith and then navigate you through the UI on the surface level and then I will implement it in one existing project and and after implementing it we'll see the traces as well as other functionality of the Lang and At Last I will show you what is the pricing and the road ahead of Lang Smith let's get started okay this is the website of Lang Smith as you can see here get your llm apps from prototype to production I will by the way provide all the links of the things that I will be going through in this video in the description so feel free to go through the links in your own time okay so Lang Smith is a unified Ops platform for developing collaborating testing deploying and monitoring the llm applications right when you have the llm applications you need to have all these things if somebody provides you one place to do that then this is really good right so as it is shown here also before the 80k plus people signed and 100 million plus traces logged in 5K monthly active users and yeah this is the life cycle as you can see we develop things we collaborate we test we deploy we Monitor and it keeps on it keeps on going right yeah you can just go through this website and see in depth information but I want to show you this four things here the main thing of langage Smith in on a higher level is there is the traces that you can see there is a hub there is annotation cues and there is the data set I will show you in the demo part how this looks like in the Lang Smith website itself if you scroll it bit down okay these are the main things as you can see you can you can see the user feedback uh collection at Advance filtering online auto evaluation spot latency spikes this is really good when you have a application let's say in production you want to know what is the what is the latency uh things right and also the inspect the anomalies and errors you want to track the cost and so on right yeah langage SP turns llm magic into the Enterprise ready applications and by the way some of the questions I I really find this website helpful because they have this got a question of course there is a question because is it is the company data is let let's say is the data being used or not right you can go through this but I find this really helpful where is the data stored traces are stored in gcp US Central One organizations traces are logically separated from each other in a click house database and encrypted in transit and at rest I think this is really important topic they have mentioned here and also the next thing is let's say that will Lang Smith add latency to my applications no it will not add that is another good question here will you train on the data that I send to Lang Smith right no we will not train on your data and you own All rights to your data you can see the terms of service this is another good topic and also how much it cost yeah this is the high level things what is Lang Smith now let us go to the Lang Smith UI this is the UI as you can see here but you need to be logged in I am already logged in here but for that how it looks like is if you go to this smith. langage smith.com if I open this in a Incognito you can see you need to of course sign up before you would have to be in a waiting list now you can just sign up with any of the things available here so yeah just on a high level now let me show you what is in the UI there will be the project section imagine a project let's say you are creating an application you want to have all the traces from that applications into one project that is on a high level I will show you this when we uh create this and there is data set and testing meaning that let's say you have your llm traces you can create that into a data set and then test also how it performs that's the simplest possible way I can explain this and annotation cues add human feedback to your llm projects I will show you one simple example how you can add let's say data set into annotation queue and it's up to you if you want to go into depth or not because not all the applications need all all the functionality of Lang Smith and at last there is the Hub where there is the prom s already available prompts or you can create your own prompt and share it with others so you can use it also so yeah this is the high level things and let me go to the notes I explained you what is langage Smith I navigated you through the UI and surface level now how to implement in your project let's go through that okay this is the main website as I showed you before also right so we will be creating a new project here and then using the environment variables into our existing project so I will go to this new project here I will create a new project or you can just give the default name here and just submit it is created a new project here with the name given by default and there is no llm calls right now there is no Monitor and there is the set of things here so as you can see here this is the thing that we need to copy right I can just copy this now and go to my vs code where I have opened a project here so this is the simple project that I have already shown you before also the L chain AMA chain lit and yeah I'm going to just run this simple application here and there is this NB that I have already used but what I will be doing here is just replace this with the one that I just copied and I need to remove this export from here because that is when you run that in the terminal so I will just remove this and now one thing is we also need to replace or place the API keys right from where to get the API Keys If you go here in the Lang Smith website there is this API Keys click on this one and create a new API key so this is one which I have already created I can delete this so this is a new one copy this I'm showing you this because I'm going to remove this once I create this video so I will just go here and just contrl V I will save this one now right so all our environment variables are being set and in chain how how it works is when we have the environment variables in the EnV file the the app will automatically get those environment variables I'm just having a simple chat UI where I'm asking I'm giving the system prompt you are a machine learning engineer who knows most of the machine learning steps right the main idea here is not to go through the answer but how the traces appears in the Lang Smith website so yeah just to go again back to the website so if I go again to the projects and if I go here which I just created so this is the one uh it is just created right this uh granddaughter 99 so there is no llm calls there is nothing traces right so now I will go to my terminal this is the Lang chainama chain lit so here I have already activated the virtual environment and install the necessary packages so I will just go here and run chain lead run simple chai UI Pi there is a typo chat UI but you you get the idea so I will just run this here so as you can see here it says that loaded Dov file meaning that all the environment variables are being loaded automatically so I will just go here and say what is ml for example right so it is quite fast I have shown you this example before also and now this must be let's say traced the trace of this llm call must be in the L Smith UI right now as you can see here this is the grand daughter 99 and as and it is just shown here this is the question that I asked what is ML and this is the answer being provided right so you can ask as many questions as you want and all the traces will be shown here so this is how the traces are being shown in the in the langage Smith in the next part what I will be showing you is for example going to this another another traces which are I have already asked this Lang Smith check and I will show you the extra functionalities what Lang Smith has to offer so you click on this one if you click on this one so it is going to open let's say the new window here so it will show you the calls as as is shown here runnable sequence and if you click on this AMA it is going to show you the output here if you go to this runnable sequence so you are asking the question what is machine learning and this is the output right and you can even choose from here what type of answer you want to get for example this ISL this is Json and ml right so and you can even give the feedback let's say that you are okay with the feedback then you can also get the feedback so there is no feedback recorded for this run as of now now if you go to metadata what will be there is all the things for example we are using the AMA llm and then the other metadata is Lang chain version this is what it is being used library and all the different things so just play around with this because sometimes if you you need to have the latest version of Lang chain and now let me say that you want to add this things into a data set that is what I was going to show you so here is the thing called add to data set so if you go here to add to data set so this is the input this is the output and you can choose where you want to add I have already created one called example AMA but you can of course go here and create a new data set I will just go here and add this into this I can submit so once this is done this is added and I can see the data set with the view the data set as you can see here I have tested different things in the afternoon also and now you can see that different examples is being uh shown here and you can test if this is correct or not with test also here I'm not going to go through test right now but if you go in the test so you can create a new test for example you can see from here there are different things and the code is also being provided you can choose any of these on the left side also if you let's say you choose harmfulness and as you can see here the code will be added here in the existing code evaluation this is really good how they have implemented this and I'm not going to go through again here as I said you but one uh more thing if you go to the projects if you go again go to this one as I said you if you click on this and here this is theats icon here so if you see it takes 3.52 seconds and this is 24 prom tokens 79 complete tokens and also for others also right it is shown here and and by the way if you are using the paid version for example I think it is now supported in the in the in the open AI models that you can even see the cost associated with that particular particular thing so if you if you just uh scroll this here as you can see here it is added in the data set we just added that right and if you scroll here you can see how many tokens and first token how many times time or millisecond it took what is the cost because this is free it is not shown here and if you scroll here there is the metadata reference and if you want to add more if you see here there is this columns if you click here you can choose which one to show in the U now almost all are being chosen here and also one good part is let's say that on the left side also it is shown the details for example here this is the run count is three and there is 420 tokens being used okay median tokens error rate is 0 % is streaming 100% latency you can already see from there and filter shortcuts for example if you just want to see something also let's say I just want to see the change and it will be filtered based on that and then runable because now we all have that but if you have many let's say llm calls or traces then you can just filter out from this filter things these things is let's say that you practice this once you let's say go into into into implementing things right it's not that uh you need to learn all the things at once but just explore these things now it is available for us all right this is one thing and also this is the traces if you just want to see the llm calls you can just see the llm calls because I'm using the AMA model as you can see here it is showing the AMA this is the system message start time what is the latency data set as I showed you before all the different things are shown here if you want to go inside click this and it will be showing here okay runable sequence and you can even go here and see okay what is the stats from here most relevant all as you can see there is the string output and yeah I don't need to go and show you all the different things but you can even go and share this that's another thing and one more thing that I want to show you here is if you maybe let's go to the traces now there is because I explained you the data set part I showed you the traces and the third one was the if I go again here there is this annotation cues all right what is this annotation cues if you go here it says start by bulk adding runs to a queue from a project Right add human feedback to your llm projects that's what The annotation Q does and let's say that you you want to add some of the llm traces into this que how to do that you can go to the projects choose the project that you want for example this one I will choose this project and here as you can see it is add to data set we have already add added to this into the data set and this is s to annotation Q if you click this one so now it will are it will ask you where to send there is nothing now so I can create a new annotation queue I can just give this test I don't need a description now I will create this one and now it is being added to The annotation queue and the test so here someone will go and check okay what is the question and what is the answer and you can even go here and change different things and you can even view the Ron from where it is coming from right and the good part here is now as you can see there is the feedback the person who is going through this this can just go here and say okay I will just give the it is correct I will give one feedback so the feedback is now recorded and you can even give some notes here that is really good and by the way you can even create a new TX out of it let's say that this is because you you can have as many tax as you want for example here there's faithfulness score you can just go here and give the scores you can go here and add more TX as you want in the project so this is really good because having human in the loop kind of things into the tracing application is really good concept so the last one I want to show you now is if I go to this hop meaning that the prompt hop right or the prompt what can I say this this is the place where you can go and see the existing prompts from others and then also you can create your own prompt and share it with others that's really good concept here and you can go and check with the use cases also there are different use cases as you can see here and there is the type look chart prom template or string prom template different languages also and what are the different models being used right so this is really good for example let me say that I just want to make some joke uh joke kind of things right I will go here and type joke so it says topic joke generator I will go inside this and this is The Prompt and what are the different comits made by the people we can also see this because this is the behind the scene it's the git concept and here is the prompt and can you please tell me a fun story about and this is the topic right if you go here you can just copy this and use it in your application but we will maybe let's let's try it here right we can try it in the playground itself so there is the thing called try it here let's click on this one so now there is the prompt playground being opened right right so this is could you please tell me a fun story about I can just go here and say maybe machine learning right machine learning and now one thing is there is no open AI model here we need to actually use the one from the existing one there are some fireworks Google palom and chart fireworks chat Google pal free but there are others which you need to also pay but without that you don't you cannot because let's say that I go to open AI there is the models that you can choose I can go and choose okay PD 3.5 turbo instru I can change the temperatures here and all the different things but if you want to now start it will say there is no API key so how to provide the API key already in the prom template now let's just go to our open a website I will create a new API key create one and I will copy this by the way I will delete this after this video so no need to mention in the comment section that I exposed my API key I will go here in the secrets and API keys I will go here control V enter and now if I run it again as you can see here this is AI s here is a fun story about machine learning and it provides your answer meaning that you can take the existing prompt already from this Hub test it in the prompt playground already and then use this in your application this is really really uh use case here so if you don't want to save your API Keys just go here and then just remove this that's it it's gone now if I again start this it will say there is no apid you can just quickly check these things is there anything that I miss let me go to the projects or let me go to the main website okay I talked about the projects I talked about the data set and testing I talked about The annotation cues I talked about the Hop so what's next let me go to the my notes here so view the traces as well as functionality okay the pricing and the road ahead right for that also let me go to the blog here I think I think there is something here okay this is the blog post announcing the general availability is quite difficult to pronounce of Lang Smith and our series a laid by SEO Capital as you can see here you can just go here and see all the different things mentioned but what I want to show you is if you scroll all the way down all the way down there is this the road ahead right there is going to be a support for regression testing ability to run online evaluators on a sample of production data better filtering and conversation support and easy de deployment of applications with hosted Lang Serve by the way there is think of Lang serve also in the langen if you don't know that is really good way to deploy the applications Enterprise features to support the administration and security needs for uh larger customers because the big organizations wants to have the security features there right nobody wants to just use this although they don't they say that they don't use your data and yeah about the pricing that is the main thing people talk about right if you go here in the website place so this is the pricing they have now so for example for us who just have a hobby to teste different things it is free always so there is 3,000 traces per month so designed for hobbyist who want to start their Adventure so so this is always free and if you want to have the plus that is 39 per user per month everything in the developer plus you can see what is there and there is this Enterprise custom things you need to of course contact uh the L Smith or Lang chain peoples so yeah that's all I want to show you and by the way there is also the Lang is Smith documentations where you can just go and explore different things so there is user guide set up pricing self hosting things tracing things evaluation monitoring and the prompt top I talked on the higher level and one more thing because I want to give as much knowledge as possible and share what is already existing there and there is this video called Lang Smith indepth platform overview and there is anus who is the co-founder of Lang chain he has explained in depth in this particular video I will provide you the link or you can just go to Lang chain YouTube channel and watch this this is really good video Ive watched this before I explain you here right I already have the knowledge but then I gain more knowledge and I get that okay there is something new I can share with you that's why I create this video in short so yeah that's all for this video I hope you learned something new and now you will use lank Smith or maybe because there are other tools also out there but I find myself Lang Smith being the easiest one to use and see the traces of llm calls thank you for watching and see you in the next video

Channel: Data Science Basics

Views: 3,714

Keywords: openai api, chat ai, large language models, llm, chat, langchain, langchain demo, langchain tutorial, langchain openai, langchain explained, framework, openai langchain, what is langchain, langchain chat gpt, langchain tutorial python, llms, chat models, prompt, chain, agents, langchain use case, huggingface, web framework, open source model, langsmith, langsmith in langchain, langsmith for beginners, llm evaluation, Langsmith, LANGSMITH, langsmith tutorial, flowise

Id: FgG-trkAMwU

Length: 21min 47sec (1307 seconds)

Published: Sun Feb 18 2024