Build Blazing-Fast LLM Apps with Groq, Langflow, & Langchain

hello everybody Welcome today to our live stream uh with Glock uh Glock that's the way to start grock right how we start with the correct name of it but with grock Lane chain and Langan flow and you know we see a bunch of people are already in we already have 105 folks that's awesome um we'll give a couple minutes let everybody just kind of settle in hello hello hi misba hi HJ hi Lance hello David hey David hey let's see while we're you know while we're waiting just a couple minutes um let's see oh thank you hos I say we're all here because you guys are awesome that's that's a great way to start it right I like this hello excited to use to to use and learn how to use Lear Lang flow nice to be here where's everybody from like we can see you in the chat go ahead and uh let us know where you're coming from I'll start I'm I'm in Orlando Florida right which is Way South Southeast in the United States if you're not familiar oh I see pelto Oakland oh from Brazil Berlin Geneva India this is awesome already Greece like turkey shout out to Turkey nice oh wow BOS here Uganda Toronto denisia sa Paulo this is amazing I you know I've I've done plenty of these live streams before and you know these days I think it's pretty normal to get a pretty Global uh you know uh group but this is I think one of the most varied groups I've ever seen from all the different areas it's super cool wow amazing awesome well thank you everybody for that all right we're you know I think I think we can just get going you know so here let's just you know quickly look at the overarching theme right so like I mentioned earlier um today we're going to FOC on three parts of an llm stack or an llm application stack right again um grock Lane chain and llow so grock provides some of the fastest inference uh models out there uh TJ is going to tell us more about that uh Lane chain if you're not familiar with them uh you know they're probably one of the honestly one of the top libraries to build um you know gen apps today um and they also provide grock Integrations and they make it totally easy for uh llm application developers like Us in Python and JavaScript to call the grock API and then langlow makes it super easy to build grock based llm apps using a drag and drop uh approach you know it's a no code kind of UI approach and that's also built on top of python and Lane chain so all these come together all these three things they come together to provide a really neat stack that you can use to um accelerate your application development okay so before we get into some of the core stuff just a couple housekeeping items I'm going to share my screen here um oh I see a thing am I the only one with a black screen or is fetching stream for everyone yeah that's a great question actually um do me a favor can everyone give a thumbs up let us know if you're seeing the stream okay I see a couple says still waiting I wonder if that's just how um crowdcast I know does take a moment sometimes I think it'll catch up for those folks okay excellent thank you so much all right so what I'm going to do here real quick is just a quick housekeeping item so while we are going to have some content and we're going to be sharing some things we do you know we're looking for this to be more interactive we're going to have some things for you to do so you can see on the very right hand side over here the the messages that all of you um are interacting with now but if you notice you'll see that there are both the Q&A section so if you've got questions during the session the best thing to do put it in the Q&A then everybody else can vote on those they'll they'll Bubble Up and you know we can we can manage answering the questions as effectively as possible um we'll also have a couple polls today um so we will show those when it comes time um but just so you know um you come over here to the rightand side to interact with both those things alrighty and with that I will uh pass it over to hati uh to go ahead and give us the skinny on grock awesome thank you so much David hi everyone um super excited to be here and to see so many of you from all over the world uh my name is hatia ozen and I am a senior customer applications engineer here at grock uh a little bit about me I've been at grock for about two and a half years now and I wear many different hats here but through the ever changing hats uh one constant is my focus on developers developers and developers uh devs just know best and I'm always looking to enhance the gr developer experience with my team and just being the developer Advocate to our internal engineering teams as well I am also in charge of our quickly growing community on Discord and we'd love for you to join us there for continued discussions after this awesome event with that uh let's dive into it starting with the big question what the heck is grock so for those of you who may not have heard of us yet um or if you've heard and you don't know too much Brock simply put is fast inference so grock was founded in 2016 by Jonathan Ross um who's the creator of Google tensor proster unit and we are headquartered in Silicon Valley but we have Grocers all across the world right now we are an AI infrastructure startup that builds the world's fastest AI inference technology known as the lpu inference engine the lpu inference engine lpu standing for language processing unit is a hardware and software platform transforming the pace predictability and performance especially performance as we will see shortly of AI applications our lpu inference engine is responsible for the world record-breaking large language model inference speeds that you may have seen in the lead on Independent benchmarks such as those done by artificial analysis and this insane performance is all thanks to the architecture of the grock lpu that is very well suited for computationally intensive applications with sequential components such as the llms that we know that we're getting to know and should I say that we love so much today and I won't just tell you I'll show you uh we were just talking about the developer experience and how important we think it is here at grock today I want to introduce how to get started with llms hosted on grock through our chat UI and through our grock Cloud developer playground using grock API so I will go ahead and share my screen and you are more than welcome to follow along with me okay so hopefully everyone should be seeing that but um the first link that I want to go through is gro.com so this is our uh UI for our grock chat um and here if you're following along with me let's sign in uh to be able to query and try out our fast inference speed for yourself you can create a free account or you can log in if you already do have an account and since I am a groer I do already have an account so I will go ahead and log in and once you are signed in you'll see at the top right there appears a list of the llms that we are currently hosting on grop cloud so right now we have Gemma 7B we have the meta llama 3 models available uh both the 7D billing parameter and 8B and we have AI mixol 8X 7B so we talked about inference speed I will go ahead and just very quickly show you what I was talking about and we can do AR grck Lang chain and Lang flow so exciting and there we have it folks we have more than 1,200 tokens per second that is instant instant response and I just want to take a moment um for this audience in particular I know there are a lot of developers out there globally just think of the speed Beyond just a simple query what this can unlock for AI applications in real time uh one example that comes to mind immediately is of course chat Bots having a near um humanik experience with high inference speed and low latency um is very very important important so with that this is just simply our gro.com which has admittedly um replaced my regular search engine in my day to-day uh where I'll just come and ask gro.com my questions we can cut over to console. gro.com so if you're following along console. gro.com once you get on the landing page um you will be once again prompted to create an account for free um or log in if you already have one in my case I'm just going to go ahead and log in with Google and then we arrive to our developer playground so this is a little bit more interesting here as a developer you have access to more granular features we have our very important system prompts we have to the right um again the list of the models including parameters that you can adjust to your like for your AI applications um such as temperature max tokens you can enable and disable stream mode Json mode for your applications which is really exciting and here I'm just going to demonstrate um for example the system prompt helpful assistant uh that always answers an very upbeat and fun um tone and then real quickly what is grock in three sentences oh boy let the llm tell you all about all about grock but um with that I just want to demonstrate if you're experimenting and you're testing around on the playground and you happen to like your system prompt and the uh settings that you have you can very easily just go to this view code button um and get the chat completion code um copy that to your clipboard in language of your choice and just simply put it right into your AI application so here we don't have too much time together so I just want to highlight a couple things one we went over creating a free account two and the main thing that you really need is generating a free API key so this is really exciting actually um for us at grock and uh hopefully for you developers as well we launched grock cloud and grock API a few months ago um and we have been in our free beta where we are you know just seeing hundreds and hundreds of applications from our now 250,000 Global uh developer Community uh just creating apps tutorials videos um all thanks to the uh free B dat that we have and the API key that you can generate very simply and with that if you're thinking okay I've experimented on the playground I like what I'm seeing with inference speeds we have the comprehensive documentation and quick start available for you here and I do want to give a shout out to again developer experience and just things that are created by devs for devs if you have existing applications maybe you're using open AI opening API right all you really need is the base path that we provide switch out for a grock API key choose a Gro API model and you're good to go for switching out the back end for running on grock so we are trying to make this as simple as possible for everybody as we're mostly open a compatible and just a couple highlights here that I want to give as well we have the documentation available here but for those of you who may be beginning ERS all the way to Advanced developers um that are maybe you know starting with students dipping your toes into the world of llms and all these technologies that we have available uh we have what we call our grock API cookbook that I will show here on GitHub um where we have from our internal engineers and external developers just from our developer Community uh contributing these very comprehensive guides where you can learn everything from function calling to how to implement um a retrieval augmented generation workflow using grock so just want to give a shout out to that and with that I think the final uh pointer that I want to give all of you we will be hopefully answering almost all of your questions uh during the session that we have together but uh in case we can't uh we also have our Discord developer community that I mentioned at the click of a button you can join and talk to me in my team there um or you can talk to our team via this live chat feature that's also available on the gro Cloud um developer playground and just to quickly recap um with the simple steps that we went over we went over gro.com and the chat UI that we have we went over our grock Cloud developer playground and all of the resources that you can use to get started and we went over how you can just very very simply today right now generate a free API key to get started um with building on Gro and I'm super excited about the following sessions as well where mispa and Lance will show you how to do that using other Technologies so really excited to see what you can build with that David that is all I have awesome thank you HJ so by the way I think we have a couple minutes here left um which we can address some of the Q&A questions and there was one it totally related to what you were uh what you were talking about here um from Chris Collins the top rated one what is the time frame for grock paid so we can get higher limits on the API and this is this is the top question um from the developer Community actually um at the moment I don't have a timeline but stay tuned in our Discord developer Community Channel I believe we are coming out with Enterprise plans on July 1st but for Vel opers um Chris anyone else on here if you have applications that require higher rate limits we have been partnering with developers like you in exchange for demos to put on our website um for higher rate limits so uh if you contact me um I will be happy to kind of discuss your use case uh discuss your application and do something for you hopefully awesome thank you and you know we're going to switch to a poll real quick I do want to point out if you look at the bottom of your screen you should see a little call out there it says check out and contribute to our grock API cookbook if you click that that'll bring you um to that content so you can go check it out now before we move it over to Lance uh we did want to ask everybody something uh a poll here that if you go to the little poll on the right hand side you should see a poll that says what kind of llm apps are you building um so we'd love to get a feel real quick uh for what kind of llm apps You're Building I see one oh here we go here we go a couple of them coming in wonderful oh look at all those rag ones and a lot of AI agents that's super cool actually nice I think everybody's built the chat Poots now super usedful all the chat chat Bots that could have been built have been built but honestly you know I I thought we thought about putting an option for ragot because that I think now having a lot of the chap Bots that are augmented with rag is actually super powerful um and I'm seeing that all over the place but I'm I think it's awesome to see a bunch of AI agents um now if you answered something else and if you're willing drop it in the messages we want to know like what is it that you are you're building if it's not one of these things all right awesome that was a lot of great interaction excellent all right with that thank you everybody for answering the poll and now we will move it over to Lance to dig into Lane chain so go for it Lance yeah I will uh go ahead and try to share my screen here yeah so that was a can you guys see my screen go slideshow here um cool it's sharing now good yeah so we we've actually been big fans of grock uh since kind of they they launched and I'll tell you a little story about what I've been working on with with actually Gro and and also meta so we've seen really high interest in tool calling with open source llms and a few of you mentioned you're working with agents and of course tool calling is very Central to uh building agents so let me just first explain kind of what is tool calling so if you have an llm you can actually bind external tools to it to give the LM awareness that this tool exists and this is one thing often kind of confusing about tool calling it's not as if the and this is also interchangeable with function calling but the key point is this it's not as if the LM has some magic ability to call this function again it's an LM it's string to string but has the ability to know this function exists and to return to you the payload needed for the function so given an input question you can bind a tool this is with Lang chain you can use this little tool decorator bind a tool to the LM and if the question is relevant to the tool the llm can then decide okay I should return the output necessary to call that tool so it returns to you the name of the tool to call and the input of the tool based on the based on the question so that's like the general General uh kind of flow of tool use now what's cool is grock has tool calling available with various open source models and I was actually working with meta on some recipes and we were looking around we need a tool calling Lama 3 we saw that grock had it available and actually had very good performance from our testing so we've actually used grock with tool calling and Lama 3 in a set of Lang chain agent recipes that we've put out actually in meta's uh llama recipes repo so Gro is very very heavily featured here uh really because it just works um and we found tool calling with grock and llama 3 was quite good so in this case this is all you need to do this is in Lang chain of course I'm importing ch gr and of course I'll share these slides and you can see the notebook down here um just have to import grock Define chat grock specify your model and then you have a prompt you Plum that to your model your llm here and you bind the tools you define so that's a general flow I'll share the notebook below but the main idea here is this is actually Central to enabling agents this is kind of the typical agent architecture that a lot of folks have been using with Lang graph so this is a library that we have that basically allows you to to lay out workflows or control flows um and the way this really works is is actually pretty simple all you do is you define basically what we call an assistant node so that is your llm with bound tools and that's Gro in this particular case with llama 3 um that receives an input from the user and then the output of the llm is either a tool call or it's just a natural language response if it's a tool call we have with this this tools conditions node that process that information and routes it to what we call the tool node which actually runs our tool and so in this particular case the input is like what is Magic function 3 that grock in this case will return okay here's my input here's the magic function and the tool node will then process that and it will return the result back to the LM LM reasons about the result and returns a natural language response so this is kind of like the canonical way can layout agents um using Lang chain uh Lang graph and in this case llama 3 with grock with tool calling um so the notebook itself is here and I will share this in the chat very shortly I'll let you play with it and look at it but basically we show how to take llama 3 and give it access to whole bunch of different tools we use replicate here um we give it access to um you know web search we give it access to text to image we give it access to image to text we give it access to text to speech all through replicate and we can't Define that as a tool list and here's the key thing we Define our assistant we're just using grock here we Define our assistant here we bind the tools to our to our llm and then we lay it out as a graph kind of like we show up here that's our graph and at the bottom we show a bunch of examples of this working with many different tools so anyway that's like the big picture um it is really easy to get started with tool use and different open source llms using Gro um if they're willing to talk more about their implementation for Tool calling I'd be really curious but of course maybe that's proprietary um but I'm going to share this notebook right now and maybe we can just kind of segue into any questions that folks have um and also Mo move on maybe to the next one and then kind of come back um so um maybe I will go ahead and let you guys continue and I will share my slides as well right now yeah no worries and you know while we do that I think we can get to some of the other Q&A I didn't see any specific Q&A questions come up um for um Lang graph or Lang chain um but there are still some of the gr ones I think we just go down the list a little bit there was some uh conversation in the messages Lance um there might be you might have some thoughts on some of those um but I'll go ahead and I'll just address some of the um or ask some of the Croc ones and we can kind of get through this list here a little bit um so I have a grock account and this is from Enrique I have a grock account will using the cheat client on my developer account take away from my free API credit no so we are um completely free right now uh in the future if we do Implement credits it's nothing is taking away from anything right now so please uh play with grock API build on it to your heart's desire uh the only call out um we'll have is that there are rate limits due to the vast number of users all righty yeah makes sense we we did over here with uh our Vector database and Ne we did very similar things actually it's like go at it but you know we have to control things somewhat to make it reasonable right like yeah absolutely um let's see there is um another one here this is please recommend tool this is a I think a little deeper one maybe please recommend tools for comparing llms I wanted to compare output latency price token bandwidth so on and so forth now I'm going to float one really quick which is going to be funny which mbba is probably going to get into a little bit but langlow it's an open source project if you don't know about it already you will a little bit here in a moment It's a Wonderful tool for that because it'll allow it's it's model agnostic it'll allow you to wire up multiple models all at once and test them out as a matter of fact I was actually just doing this myself yesterday I've got a um a language app I'm building um to translate any language in real time kind of like you know um if you've ever read hit Shack your sky to the Galaxy they have the babble fish you just put it in you can understand any language and funny enough uh uh tjs you were talking earlier about the grock models and that that kind of deal I'm using those because I wanted to test and compare against open Ai and against mistal and against whatever and then it ends up being a really nice tool that I can just wire them all up in parallel and I can see what the outputs are I can see their performance things like that but that's a really bias answer do any of you um have any tools or anything that you might use to compare llms uh and we we commented this as well but um the the one I mentioned before that comes to mind is artificial analysis they're in dep dependently benchmarking um some of the the output latency price token bandwidth um so if you just do uh I can provide the link actually in the chat but if you search up artificial analysis you should be able to see uh all of the the comparisons that they've made between providers yeah that would be great actually and I think that more directly answers s sano's question there um so yes please yes please let's see um now I think this got answered in the chat um and this is definitely more of a grock question uh maybe a stupid question does the speed hamper the quality of the output or how does it affect it compared to standard LM inference so the speed does not um affect the quality of the output at all we are simply taking uh these models and running them on our grock lpus which just happen to be really good for llms um so no there there's no uh there's no hindrance to Quality and there's nothing that we do um to really affect the quality if you're more interested in the uh quantization aspect um I also have a paper that I can link in the chat if you'd like to read about it you just came with all the material today and I would hope I'm you know I would know these things being here like two two and a half years I was pondering on that startup World it feels way more but nice all right awesome yeah please that would be great um so what I'm going to do is I'm going to move into the next poll um this one will be have you deployed an llm app to production I am personally very interested in this we kind of joke here that this is like the year of uh you know production AI because I think I think since last year right we've all been figuring all this stuff out right and like doing lots of tinkering working with all these models how do I build these apps but then taking it and actually pushing in production and doing something real is is a whole different kind of ball game and so we're curious and so far yeah we see some folks have no just playing around there's a bunch honestly it's a bit it's it's a little more even across than I thought it was going to be all right well we'll give you all a minute to answer those and in the meantime I'm going to pass it over to misba uh and misba is going to dive into langlow with it have that MBA awesome yeah thanks a lot David yeah excuse me so yeah my name is misah I've been um working and uh I think to start with as as Lance said we are big fans of of Gro and also we are big fans of langin here at langlow so I work as devil here at langlow I've been building content U around how to use no code tools and how to build llm apps uh using no code tools and also with the code so uh have some content how you can use Lang chain um and and with that uh started focusing more on on both sides I would say you know as haa mentioned uh on the developer side uh but also on the on the no code developers right so on both sides of things and uh it's uh it's it's been fun Journey so what I'm going to show is real quick how one could get started with langlow and from there just want to showcase how you can drag and drop you know bring in these different uh providers especially how we can bring in grock's uh um module or node and then call different models in there and from there we'll look at a couple of examples so yeah let's let me share my screen real quick okay just a quick check if you're able to see the screen okay awesome all right so um Lang flow what is llow right for people who are just starting or first time uh learning about langlow llow is a tool that helps you drag and drop and build a an llm app completely no code if you like but then also there is an option it's it's built on top of python lank chain so you can pretty much code and write python um Snippets or code in there um and also build code based apps if you like and how do you get started well first thing is you go to this repo you also see a button down uh at the bottom of the screen which says star length for repo so that takes you to this um get to repo the easiest way that I've seen for people who are on the code side of things who have played with python or so you can um just start or make a directory in your computer or anywhere you're running this um start a virtual environment and then just pip install L flow the easiest thing and then after that you can run with this command so once you run you'll see the L flow UI other aspect I wanted to cover is people who don't have coding background who just want to get start quickly you can click on one of these links so cloning this space it will open up in hugging face and then you can run the uh langlow UI over there as well uh some other folks have seen uh it makes it easy for them to either use Railway render you click on these buttons and it takes to those respective platforms and over there you can basically select um a few options and then I think in a minute or so it will deploy the application so once you deploy the app how does that look like it's basically something along these lines where you have the the the my collections um screen so this screen is where you see if you have previous projects then you see them here if not you can create new projects and each project is where you can uh drag and drop and build LM apps as mentioned before so I'll quickly go over that uh it seems like many people have have built these apps in the audience so uh just want to kind of go over Basics real quick and then we'll jump on to examples so the way you drag and drop on the left side you see these options everything starts with some sort of input so usually I tend to go with either chat input or text input and then uh with this you can provide some you know text here something like hi there and then what happens is if you run this you'll see that this is available it's a check mark you is available as an output that you you can connect with something else and what I'm going to do is next step I'm going to call grock and in here uh what I did is I took the API key that AIA showed before and I just added in here as a variable so you can add as a variable or you can just type in um the the API key and then once you have that you can select a model um in this case I'll just select llama 3 8B and then I'll just connect the text input to the input coming into this block and with that if I were to just run this I'll see that it will send the the question that we had or chat we had to the model and we get certain response back so now this response is basically back from one of the models that we selected so once we have this we can send it out as a an output from this this uh particular flow so what I'm going to do is I'm going to either select chat output or or maybe a text output um and then bring that in so this is a message so I can just bring that in as a message um and then with this this is very minimal basic flow now the the cool things what I like is that there are a few different options once you run this um there is the playground you can see your input you can run or ask questions here um you can also call this API this real quick pull that okay so this is people for people who have some U coding background or for folks who are coming from any of the no code Builders like bubble or um flutter flow so you can pretty much take this URL um and then call this URL with your particular queries right so if you have certain input then you can add that to to chat or tweaks or so um it's also available as a python API you you can copy paste this code and then you can just run in any python application uh not just that you can run this uh python code as as is with you know export of the flow that that you can perform export the the whole flow um also there are a couple of other options you can embed this chat widget so these are some some Basics uh that you can run with and we also cover some of this in our webinar Series so we did a few of these uh last weeks so you can also follow along with some more details in there now wanted to maybe just highlight something real quick so I built a few you know basic demos right to to get started so one of that is basic prompting so in addition to what we had in the previous um uh flow what I did is I just added a prompt in here a template and in this template I'm saying just take the user input and in addition to that um answer like pirate right um so what it does is anything that's in this curly braces it's going to be a variable and it allows you to connect something to that variable and it takes that input from the user so I said hi there and then I have the user input going in and based on that now if I were to run the whole thing what I'll see is it will um see if it calls let me just read refres this real quick so basically what's going to happen is it will take the the the user prompt and from there okay let me just actually yeah perhaps uh I'll just that whole thing so at the end what's going to happen is it just um calls with the prompt template that we provided so that's one example as it refreshes we'll we'll go over that so now this is basic right so some people you might have worked with agent applications or you want to call the tools or so uh that is also something available in in the interface so again going back to that example I'll just run this something and then we see over here okay now we responds and if I say hi there it responds back you know in the pirate language cool so we can do more things to it right what I'm going to do is I'm going to ask to write a blog and again taking the promp taking some info from URLs and in here I'm saying I'm going to take the L chain website uh maybe dogs from Lang chain uh maybe also take the GitHub repo and what I'm saying is just to write uh about Lang chain taking the sources provided here and the prompt template also has some references and and instructions that we gave if we were to run and and something again uh if you're using Gro you know the the speed of things that happen is is quite fast as okay I have some trouble I think from from just the the lag that I'm seeing right now with uh with live stream or so but then at the end you'll see things will happen real quick so let me just rerun again and then it builds yeah if you look at this step just pretty much just some you know less than seconds or so it even tells you how long it took or actually two seconds in this case uh but it tells you that okay you know based on the info you gave it wrote down and it gave the the detailed blog post over here right same thing you know I can also look at that in in this screen okay so now one step up I said okay how about we just write you know not just a Blog how about we just write a full book right um that is also possible and since you know things are quite fast using Gro what I did is I took an input of text said okay you know I want to write a book on LM Warrior just random random title right so with that I'm saying First Step take the book uh title write five um chapters to it and then send it to to you know one of the models um that's hosted on Gro and then get some output so when I run this what's going to happen is it's going to get some of that um yeah I mean based on on what we asked it gave us five titles okay nice now I'm just doing some tricks over here to send send each um of the list variables to uh to models to write chapter separately this is the simple way now the the versions that we have coming out um they also have the the for Loop capability IFL so you can combine pretty much all of this into just one block or not even that you know even just the the block that we using before so go check out some of the latest versions that we we got out for Lang flow it's got some some cool stuff so with that I'm sending each title uh chapter name to the gro model and then I am asking it to bring all of that text together and then save it as a text file so just for fun I okay actually I'll call this something else I'll call this book demo and let's run this once we do that what's going to happen is it will go through all of these steps make bunch of calls to the model and then at the end it will come back with the list of all of the the the the the book chapter so this is my agent which write books for me okay so that's done now we should have all of that and it's saved to this file which I can go check real quick now that's you know the the full text it's uh quite large and this just for the five chapters of course you can have you know multiple chapters or you know you can can say okay write for 20 chapters or so and it will write the full book for you so again just to recap you can start with some basic um llm apps something like basic prompting of course we all did the chatbot thing but now if you want to add some additional capabilities what you could do is you could start with some template those are available here uh if you're doing rag there's some basic template available here uh you can of course swap out the models here with grock if you're using GR in there um and then you can save it to database um then get some um answers from from your um data set whatever you'd like to ask from a a rag application so again you can build um any advanced capabilities it makes it easy for you to just drag and drop and get things started yeah I think so with that I'll pass it back to you David if there are any questions yeah M actually I want to put you on the spot um you know I know that we're focused on um grock and Lan chain and everything today if you wouldn't mind if you could actually share your screen again and just point out how um in models in the vector stores you know again I I mentioned way earlier that Len flow is agnostic right and there's all of these models and the way that you can you know you can kind of test those and and bring those in the picture um so yeah go ahead sure yeah let's look at that so for example I had this Vector store rag right this is one of the templates available um in langlow if you were where to just start with with the template mention um and then not just that any any blank um canvas right you'll see a bunch of options on the left so for data for models you'll see uh Vector stores right so there are quite a few options here you can just either have a self-hosted version so some of them they offer that uh you can have something through an API um or something like pine cone then you can call that so A bunch of options avail able here same thing with with many other things you'll see um agent based um uh things as well so like tools or some other utilities so quite a few things I would say you know just play around see uh what what works for you A lot of it is brought in from Lang chain so if you're familiar with with Lang chain what it offers then you'll see that available here and in addition to that we also build something internally so uh that as well as custom components right you can always write your own code and and run anything that you can do in Python yeah and one thing one last thing I'll add with the um the various models that you have you can also pull in variations on the same model like the matter of fact I was working with grock it there's multiple models that are available and I want to compare them right so you can just you can just whip them up right next to each other compare them in line um use the same API key like it's super simple it is is just so much faster I mean I I like coding I I cat up enough things um but as yeah as by showing you you can literally just pull it out pick a different model hook it right up you'll need another chat output for that one um oh yeah so uh yeah so anyway I just you know it's it's really neat for that kind of flexibility and and being able to you know get your your AI development going all righty so with that we do have some time here at the end to get at some more of these Q&A questions there have some that have bubbled up to the top and thank you Ms by the way um this one's gotten a lot more um uh let's see a lot more scores here uh how can I effectively uh connect an unstructured database to large langu language model agents to enable efficient querying and data retrieval um and there have been a couple comments in here as well um but I'm curious um if if Lance or TJ or I'm G to put youall on the spot if you guys have any thoughts on these yeah sure um I'll share a video series here so I have a rag from scratch series on YouTube uh which I think some folks referen this talks about rag so that's kind of the first component I think we talked it was talking about connecting agents with Vector stores so rag is kind of the technique you can use to connect an llm to a vector store there's a lot of tricks for rag I shared a video series we did there it's also on free code Camp um I can share that link as well um we have a bunch of I'll share I think it's actually our most popular video ever is on um uh building rag agents with llama 3 I will share that as well this actually could can use Gro uh but it in this particular case it's using AMA for local but it shows you how to build an agent that performs rag with a whole bunch of internal self Corrections um so those some resources to start with awesome thank you Lance and you know to that one of the that viso is showing the uh the template that's in langlow the vector rag one um you know it starts it it has um you know it has the ASB Vector store but there are all sorts of others right and you can just swap them out right so in a case like that you could also wire them up that way it's you know it's an easy way to construct it and if you don't want to do it in in pure code okay let's see so the next one here Lance this is definitely one for you uh how how does Lan graph uh differ from Lane chain yeah yeah so Lang graph is basically a way to build agents so prevy and Lang chain so Lang chain is a general purpose open source library for building llm applications does many different things um it includes Integrations with llms like rock it includes Integrations with Vector Stores um it like feeds up to Lang flow um Lang graph is the newest kind of way that we've kind of devised to build more agentic applications so previously we had this thing called Agent executor within Lang chain which worked we found a lot of people wanted more fine grain control so langra is a way to build agents that lets you basically lay out your agent as a set of like nodes and edges and I shared that video I shared actually talks a lot about Lang graph um yeah I will I'll share it again just to like emphasize it but basically it's a really nice newer way to build and deploy in the next few weeks you're going to see some stuff related to deploying on Lang graph a we've seen a lot of uptake from companies the main benefit of Lang graph is it can be extremely reliable because you have a lot of control over the flow of your agent so in that video I recently let me see the the I'll share it again this one shows how to build an agent with local LMS so it's reliable enough to build a complex agent uh with local models that run on your laptop and langra kind of lets you orchestrate in such a way that's reliable um so I will share that awesome and I think that actually addresses there was another one here in the Q&A could you please speak to the integration of L flow and Lan graph um it feels like you you probably hit on that is there anything more you want to say from that standpoint yeah maybe miss you speak to that yeah sure um so I mean in general there are plans to actually um get some additional integration um that is probably going to happen soon so be on a lookout um so far in in general um if you can write python code you can bring in any of that already into langlow um and any aspect that that you notice is not in the UI you can pretty much just write custom component and and and write for that but then soon you'll see some of that integration available within uh within langlow UI as well talking about the the Lang graph and and actually the the Lang Smith as well cool okay excellent thank you and let's see we have a couple more let's see um so does Lang flow still use Lang chain under the hood to chain the components and execute the graph mostly yeah I mean that's where it started right uh so there uh I I would say you know to step back it is python based so anything you can do in Python again you can write custom components you can install any of the library um where where you're installing the LF flow um pip installation right so and just have all of that available in um in Lang flow uh in general but there is some bias towards the Lang chain Library initial things reported from from Lang chain librar so you'll see a lot of that but again both right so you can you'll see a lot of Lang chain in there but also you can bring in anything from the python World whatever you would like to bring in yeah and kind of case in point on that um you know Ms and I we're recently working on some flows and in particular in one U you know built in 11 Labs functionality if you're not familiar with 11 Labs right you can it's actually super cool um you know it's more on The Voice end where you could say Supply it they either produce and you know you can you can access all sorts of different voices um but then you can also Supply your own voice funny enough which is what I was doing um you supply like 5 to 10 minutes and it is scarely good at replicating your voice and so the the point of that is that the 11 lab stuff wasn't based on L chain or anything we were able to create a custom component right in langlow that then interacts with that API and then we could pipe any of the text coming out from llm to one of these voices right and I made it like pretty easy so again it's I think um you know yes it's mostly Lang change still um but you can it's python like MBA just said you can kind of create you know whatever whatever you want honestly if you are if you're a python Dev um let's see so the next one here um this one got ranked up is how to deploy langlow made apps now Ms you were showing some of this real quick with the apis right there's there's a set of options here um I know we do have a a little time do you want to show that real quick again sure okay sure and by the way while you bring your screen up I'm going to quickly answer another one um which is um uh Lang flow L flow is moving to new version from uh 0.6 to 1.0 is a possible still to export L LCL change from Len flow to the new version um yeah I was asking in the background yes yes that should be that should be supported okay go ahead yeah so if you were to build a flow then you'll see an option here a playground and then API right so once you click that you'll see the options to call Api so something I have built content around before is also how you can connect something like bubble or flutter flow or any sort of you know front end app to the this as a as an API so you can just have your langlow app um working as backend API for all of the llm aspects and then your app whatever it is web app or mobile app or so it can do all their things and call this API for the nlm side of things so once you have this you have the URL available if it is on Local Host you'll see something like this I have another instance of this running on render and in here what I can see is if I were to click this API um it is something I can call from anywhere you know so this is live on internet I can call this and with this you know there are a few different options you can call from a python based app or from anywhere else um I can go into the tweaks I can change a few settings so let's say for example instead of calling the model that we initially called I want to change that to maybe you know 70b option here I go back and I'll see that in my tweaks that I can now call that as a parameter uh if it is URL I can add multiple URLs and I can see that option and then bring this over if you're using something like Postman or so then you can import curl and it will give you how you can call the API hope the answers yeah and by the way there's a whole chain of questions in the chat right now about um using uh deployed L flow along with something like streamlet absolutely matter of fact I'm doing this for my own development as well um and fun enough you can actually use grock or like GitHub co-pilot and you can say something like hey um give it like what misbo was just showing you you can have you could say given this code and given my inputs and outputs I want you to use streamlet and produce me modify this to actually generate a like a basic streamlet app and streamlet is slick and it it's super easy to develop in and grock has no problem with producing that kind of code it's it was actually super fun uh just recently to do that um let's see so this one um at the top can we request other models not already on the platform also for Enterprise licensing and cost uh oh it disappeared oh it just there it goes um can we reach out I think that was a question actually for grock yeah um so you are able to request I actually go through these weekly myself um if you join the Discord developer Community um the link is on gro.com and console. gro.com we have a feature request Channel that I uh check and report back to the product team with um depending on the number of upvotes the reason actually that we put up llama 3 as the next model after the first ones that we had put up is because we had run a poll for a developer community and everyone was really excited for three models in that one so we take you all very seriously um please submit a feature request either through Discord or through gro.com and the developer playground there's a little chat with us button where you can reach out directly to me and my team excellent thank you and um you know if that question by the way is directed at all to the langlow folks um I would say the Discord is also probably the best part best place to put that misb do you if you happen to have that handy would you mind dropping that in the chat um so folks know where excellent thank you sure we'll do all right so the next one here because we're almost at time uh let's see um wait here that one you know it's so funny I keep there we go this is the one I wanted I I cheated there all right so um how does Lang graph compare to other AI agent libraries advantages and Main use cases Lance I think that's maybe for you sure yeah well you know I don't want to like put down or compare necessarily against other liaries I think all libraries have interesting trade-offs benefits what I'll say is I'll be honest I use agent executer and Lang chain quite a bit previously I never found it to work that well okay so that's like me kind of eating my own dog food and saying look I didn't find it to work that well there's a lot of issues with agent reliability what I found using Lang graph is that you can build very reliable agents because you can very explicitly lay out the steps of the agent rather than relying on tool calling and actually if you look at that notebook I or the video I shared um it actually is using a local LM on my laptop to run a relatively complex multistep self-corrective rag flow and it just works so the proof is kind of in the pudding and we''re seeing this with a lot of our customers Lang graph is just nice not because it's complicated but because it's simple it lets you kind of break out your flow into a set of specific nodes connected by edges and to like break down the steps in a way that you can very easily kind of control the flow through your application and when you think about most agents that's kind of what you want you basically want to use a sequence of tool calls in Lang graph you can of course Implement that like we showed before as an agent or as an element that has access to tools or in Lang graph as a graph with each tool in its own node so anyway look at the video because it won't make total sense as I'm saying it right now but if you look at the video kind of get intuition for like lra is a really nice way to build these things highly reliably because you can lay it out as a specific control flow and the proof is kind in the pudding I've been able to build really complex rag agents locally that run on my laptop with a seven billion parameter model so I think reliability is really the benefit of Lang graph but of course I'll let you be the judge and love to hear feedback and comments on the video hit me on Twitter ourl Martin as well if there's questions but I think reliability is the big one awesome thank you goad something real quick yeah I mean using agents before and then using langra it's completely different right so L langra makes it real easy and also to folks uh if you have been watching Lance's videos on Rag and and different things if you have not you know you got to go check it out it's it's really nice uh especially the rag Series so definitely a kind of fan blowing over here yeah no I appreciate that absolutely awesome we have a couple minutes left I have a couple questions this one I will take because it's super easy to answer um are there plans to allow LFO to Lane chain code well funny enough there it's there today right so in the UI that misbo was showing you if you go to any component and click on the component you'll see there's a little code you just click on you got little little interactive dialogue that pops up there's a little code button there a little tab click on that all the code is right there you can absolutely grab that code and you can pull it right out oh looks like M's gonna show us yeah you can grab that code and you can just pull it right out um and you can implement it directly um oh I'm I'm go to a component pull out of the API and then click on a component yep there you go that one right and then the the code and it what's cool is these are wholly contained you could take that as it is pop that into its own class and and you're good to go right um so it's it's super Co now that's one way to do it as he was showing you the API earlier that's another way to do it it it really depends on your particular needs yeah and also if you have a custom python code you can just drop in one of these blocks and then you can write write any python code to it and also to it maybe we can you know real quick David so we have a u next webinar series which is going to talk about all of the new features and how you can build scalable um apps using langlow that's going to happen next Tuesday so if you are interested you know please please do join that one as well drop the link to that oh yeah please do and by the way I forgot to mention earlier down at the bottom you see that star llow repo if you're interested in following the progress of langlow please go Star us right so you can kind of keep up to date with the things that are going on um I know we are at time I just want to put everyone on the spot real quick go go around the horn um you know were there any um say key questions or things that um you thought maybe weren't addressed real quick or something you wanted to say in closing um misba I'll start with you totally putting you on the spot there now I think uh we covered some basics of course you know for the detailed um way of doing things so you can follow the the webinar series or reach out to us on Discord and we'll be happy to help over there cool how about hadj you got anything yeah same same with Misa um I gave a very brief overview um I'm sure there will be questions after this as well so Discord is the best place to reach us as well um let us know any questions future requests we will be there with an answer excellent and Lance yeah um we do a Discord um Twitter is a great place often DMS are all open and really nice to be here I'm a big fan of Brock used of course recently with the with the Llama 3 notebook so that's fantastic and missile's work onl flow is really really cool uh so good to be here and talk to you all awesome thank you everybody and we are at time uh thank you everybody for your questions and the polls and the interaction uh I saw Lance and and HJ and everybody was dropping a bunch of links and materials for all of you um you know M I'm going to put you on the spot one last time so for the questions we didn't get to are we going to drop those in the Discord actually for the the link flow Discord yeah we we'll try to do that and also we'll we'll try to follow up with some resources um via email or so to to everyone awesome all right well thank you everybody it was awesome seeing everybody here and take care we'll see you later thank you so much everyone thanks

Channel: Langflow

Views: 2,305

Keywords:

Id: 4ukqsKajWnk

Length: 61min 18sec (3678 seconds)

Published: Thu Jun 13 2024