Session 1: Course Intro | Building Your First RAG App with LangChain, Chainlit, and Hugging Face

putting them into production applications so we are cohort one we are going to be # lm1 in everything that we Post online everything that we'd love to see you guys posting to build your own brand and we are the team at AI Makerspace this is our first ever product this course on llm Ops but more broadly we look forward to building this community out with all of you today what we're going to talk about is we're gonna kick this thing off properly we're going to talk about the course we're going to talk about the different types of sessions that you can join each week we're going to talk about some semi-important stuff that you should probably know about that we will spend time on that we've actually moved around a little bit from the original idea of the curriculum and we're going to put it into slots that we think will be even more useful that's going to allow us today to get into the number one thing that I've been hearing as I've been talking to many of you and that is how do I build a rag question answering thing with I don't know Lang chain um and so we're just going to do that today we're going to figure it out and we're going to go over what is a rag what is Lang chain how do I build with it and we're gonna we're going to keep up to date with you know what's happening in the world we're going to find out if people think the Barbie movie was K for those of you that have seen it I think you'll enjoy this if you haven't you might want to check it out after tonight uh my name is Greg and my job is to sort of steer this ship I am here to really kind of introduce to facilitate to talk Concepts if you want to talk product you want to talk strategy I'm your guy and we're going to kind of Go Right Down the Line line here and you're going to meet each member of our team at AI maker space I'm going to hand it off next to Sarah our head of marketing and Community thanks Greg hey everyone it's great to meet you all I'm Sarah I'm head of marketing community at AI Makerspace I am your course support I'm your community manager so if you have questions about anything you need resources helps getting into Discord maybe a link isn't working please shoot me a ping on Discord or get my email from the from the notes after the session apart from that I'm super excited to be with you all and I'm really pumped to start learning and I'll kick it over to Chris thanks hey what's up my name is Chris and uh I will be handling a lot of the technical content in the course as it relates to llms so if you have technical questions or questions about uh some of the material we'll be covering that is not related to business strategy uh hit me up I'll be there to uh hopefully guide you through not hopefully I will be there to guide you through and uh and show you uh what what current best practices are and we'll send it to Ali hey everyone uh thank you Chris um so I do a lot of uh the stable diffusion latent diffusion stuff and I'm also here to support on the llm site as well so let me know if you have any questions Greg heiller all right so yeah these two guys are the killer coders Sarah's community and hopefully I can answer any questions that lies outside those realms um all right so what we're going to do now that you met us is we're going to meet some of each other and you know I I cannot say enough amazing things about the folks in this cohort this is the most talented group of people we have ever had the privilege of getting up in front of and trying to be useful to um this is no joke you know from from our own personal audiences from the maven audience there are some unbelievable talents in this room and you just have no idea who you might be connected with in this first breakout room um what we're going to do is we're going to do what we're calling virtual hot seats so groups of five we're going to do 15 minutes so that gives everybody three minutes to be on the hot seat so that means when you get in the room some brave soul is gonna have to stand up and say I'll go first all right and whoever does that go ahead and when you're done poporn and pick the next person we're going to put kind of a lot of onus on you guys to do stuff and to engage because each and every one of you is ready to get after it that's very much what we tested for in the application process and we have no doubt that you're going to have a heck of a time getting after it with other people that are right in the same mindset so jump in three minutes we're going to send pings reminders each three minutes for you to switch if you don't have a question that you can come up with as you're going around the room to try to Pepper the person who's on the hot seat with questions then use the random question generator like this one right here do you like to get up early or stay up late has this ever been challenging for you there's different categories that you can pick and there's different questions that you can just use this thing to go ahead and ask if you're not feeling particularly creative right now and we'll share the link with you in the zoom chat now as well as in Discord we will talk about Discord in just a little bit so with that let's go ahead and meet some of each other we're going to open up breakout rooms groups of five 15 minutes somebody jump in put their hand up to go first when you get there we'll see you back in 15 are back in two and one all right everybody welcome back we hope you had a great time meeting some new folks we are going to just keep it moving right along today because we are tight on time each time we meet in these sessions and so we're going to again honor each and every minute so what does a week in the course look like well if you joined us yesterday you know a little bit about what our Monday Community sessions are those are going to be places where we come and we share insights tactics techniques things that we are doing to build our own personal Brands online LinkedIn Twitter threads Tik Tok anybody anybody on that Tik Tok uh whatever you use this is where you know this is the thing that Ali and Chris and I and others have been doing for the last six months the last year I've tried many of these groups over the last few years this one has stuck recently in 2023 when it comes to generative AI so if you're looking to build your brand to get that next job or build that next startup marketing and sales is very very necessary come join us and you'll see what it's all about Mondays also is where you can talk to Ali oneon-one in his office hours Tuesday of course we have class Wednesday we're going to have what we're calling Community build sessions and we're going to talk a little bit about exactly what's going to be in these in just a minute but these are going to be led by Sarah our head of community although nod to Sarah she's going on vacation for a couple weeks as she very well should uh at this time of year after we have this week complete so the community build sessions will be every Wednesday night at 9:00 p.m. Eastern Thursday we'll have class Thursday I'll also have office hours at 11:00 a.m. eastern so for those of you over you know Pano or those of you over in India you can probably squeeze in in the evening and jump in if you have any questions for me additionally Juan uh is our peer supporter Juan Alano I believe you may have met him if you was in your room but Juan is basically an OG in the game we are so grateful and happy to have Juan he's a very very experienced business Builder coder entrepreneur success he's basically retired and he does his stuff for fun he's just a wealth of resources and wisdom lives in Miami and I believe we we have shared the information with onean with all of you it's a little notion sheet um and I think we're going to go ahead and find that and drop it into the Discord and into the zoom chat for you as well to learn a little bit more about onean these sessions are going to be a little bit more informal they're going to be basically just hanging out with onean sometimes you know class can feel formal office hours can feel formal even Community sessions can feel formal this is a place to just sort of hang out kick it and learn a little bit more in a more relaxed environment with Juan so Juan I don't know if you have anything in store for this week that you want to promote here but maybe just a quick introduction from you and your interest and why people should come join you on Friday thank you very much Greg um well you gave me the idea I was telling my group now that this uh upcoming peer support session will be a hot seat I loved it so we will have some hot seat there and also uh you were mentioning about my experience on entrepr ship and whatnot so I will be more than happy to share anything about it I as as Greg was saying I I semi-retired now uh after having sold my company after working a lot of years so I can you know probably share some uh tips or you know experiences with anyone that goes there so that's also another angle that we can take there if you're planning to launch a company uh maybe we can U discuss about it and and plan about it so basically it awesome awesome yeah well really looking forward to those ju um I believe the instructors are basically going to stay out of them so you know get rid of the any formality that might be associated with that if we can be of use let us know but hanging out with Juan is probably going to be a lot of fun uh we definitely enjoy it every time we get a chance to so check that out and then of course the last thing of the week Chris is the hardest worker on the team the hardest working guy that you know Friday night what does he want to do he wants to do office hours of course he wants to spend it with you cuz he's going to be building all Friday night anyway so get after it with Chris on Friday nights um if you want to kick out into your weekend with a little llm Ops so that's kind of the run through of the week and the big thing for you to keep in mind is we're going to have assignments each week we're going to do programming assignments and we're also going to do project assignments that are associated with your particular project I heard a lot of feedback from people over the last three weeks or so a lot of people doing one-on ones after they enrolled and what I heard come through loud and clear is that you know they really want to kind of see some things built before maybe deciding on a project and so we're going to kind of start the project assignments formally in week two we're going to really hurry up and get after it here in week one learn some of these really important tools that are out there on the market today get a handle on them imagine what we can build with them and then sleep on it over the weekend so that's kind of the idea the assignments are going to be due by 11 5959 Eastern on Mondays I believe this is going to work okay full transparency this is the first time that we are using Maven as a learning management platform so there may be some Growing Pains associated with this Welcome to our tool stack everyone so if you're feeling like oh I'm not into this I'm not into that Now's the Time to start hitting up the zoom hitting up people in the Discord what we want you to do is we want you to be on ramped within the next few minutes into Discord Discord I'm G to go ahead and show you what it looks like our Discord we realized Maven has a if you've taken a maven class before they have a messaging app but you know it really left us wanting and left a lot of the students that we started talking to initially about it wanting we were very much used to more of a slack setup in previous courses that we built and so we we opted to go ahead and set up a Discord and this Discord is going to look like this we're going to have all of this llm Ops cohort stuff set up for you here you're going to be able to DM everybody over here on the right looks like like we have 12 and ll1 so far so we've got a number uh still yet to join we are going to make sure you have the link in Zoom now you should also have received a Gmail invite to Discord just before the class started so this is where you're going to see announcements this is where we're going to drop links that we're also dropping in Zoom so you always have access to them this is also where you can come and drop in your intro so this is kind of what we like to start off with tell us a little bit about where you're coming from a little bit about you maybe a fun fact how to connect and Sarah's collecting hype songs from everybody uh so make sure that you share one with us that will probably come back and play a role in the community in the future this is also generally just the water cooler if you're working on stuff thinking about stuff whatever come you know smash a DM in here talk to you know call somebody out ask Chris for help ask Ali for help whatever you need this homework share is going to be relevant once you complete your homework and we're sort of and then off topic um Todd we're gonna we're gonna expect Todd to continue posting an off topic uh because Todd's the man he's the one with the sunglasses right now if you see if you're on gallery view he looks like the guy who's going to crush off topic and indeed he is going to we also want you guys to just put whatever you want in here so um peer supporters is going to be a place where we can talk to peer supporters I don't think that's necessarily one that everyone has access to but we're still getting this set up if you have ideas for channels you'd really like if you're feeling like you're restricted in any way let us know but this is the current setup I'll pause for any questions on Discord right now all right well in that case we're going to move on to the next tool which is the primary way you're going to get your assignments and this is the AI Makerspace GitHub if you go to the AI Makerspace public GitHub then you'll actually see only a couple of repos you won't see all of the reposts but if you go and you actually accept that invite that we sent to either your GitHub username or the email that you provided if you did anybody not provide a GitHub username Chris maybe just a couple of people yeah we had a few uh we had a few people not provide it so if you could post it in the Discord or in the thread or in the uh Zoom room I'll get you set up ASAP yeah so this is what you're going to want to have access to uh you will be able to do this first thing that we're doing it's a it's a public repo we're going to focus on Lang chain but then and we're also going to be able to give you access to all these fine-tuning resources but we really need you to have access to the llm Ops cohort one repo so you can get access to those assignments those are the key things that we need to know about uh third I'm going to go ahead and talk about Maven and the maven interface is is basically useful for one thing and that is looking at assignments understanding assignments submitting assignments getting feedback on assignments that's really it so when you go to the home page you should have everything you need and we'll sort of continue to populate this as we go but you see week one we're going to talk about rag with langing chain and you can go ahead and click on this assignment here or you can click into this module here we're still getting used to this in case you're not used to this so this is going to be something that we're sort of all learning together but essentially we're going to set up assignments like this each week and each class we're going to have a build component it's going to be associated with GitHub because we're here to build and we're here to code we're GNA have a ship component that means we want you to deploy this thing make it usable for others and for instance you're going to be able to look at hugging face spaces that we have set up for you on our hugging face account AI maker space we've been doing a lot of rebranding recently so bear with us if uh some of the links break from time to time hugging face AI feel free to use the link in the chat as well so this is the AI maker space hugging face you don't really need to do a whole lot with this but we will be providing access to different types of spaces from demos that we do whether it's in class or in community build sessions or even in public facing events over the next month so it's a good place to make sure that you're familiar with but it's not essential the way that GitHub Maven and Discord are and last but not least we sent out a little bit on this you don't need Google collab compute or anything like that today so just make sure that you have a Google account and you can open collab notebooks um although I don't even think you need that today and then open AI is just make sure you can hit open AI make sure you've got your key to be able to hit GPT 3.5 GPT 4 make sure that you can go ahead and tap into that API it's a great way to start learning we know many of you don't want to do closed Source building but it's the best way to get something off the ground quickly even if you don't want to expose your personal data to it uh we we totally understand and we are going to support all of you that have all sorts of problems with open AI but it's a great teaching and learning tool so with that I'm going to jump back into the slides here and does anybody have questions on tools before we move on this is kind of important and so we're spending a lot of our initial time on it yeah I don't I'm anything anything wrong but in Discord I'm I'm seeing like the LM LM Ops say Channel but not all the others um so are you seeing the llm Ops cohort one is that what you're saying yes and under that IV announcement group uh I know this maybe this is only you take offline but you know if you want to just refresh Discord uh and let me know what you see now let see I see more that just okay I see okay I see a couple others so maybe it's working yeah Chris is the master Discord mod so yeah just keep be coming with the feedback this is our first cohort and some of the tools we're not quite used to using as we were maybe in a different setup so our learning management system is different and Discord is new for US versus slack that we are used to using so if you have any issues with those definitely let us know and help us improve um thank you for that piece of feedback there which goe which channel should we ask for help in I don't want to just there's a thread Channel called cohort questions so if you just want to make a new topic and then post it in there I'd be happy to to to move through those yeah we were trying to keep it simple earlier to be honest with you guys and have just like an lm1 Channel but it quickly blossomed and bloomed into a lot more so um if you're not a super user of Discord um definitely just hit us up Chris most certainly is and anybody you see posting in cohort questions I would also put in that category so Bowen Todd Juan these people can all help you out including Sarah Ali and I wouldn't actually ask me to help you out personally but I'm kind of learning my way through it as well okay so uh a little bit of note on the curriculum this is the curriculum that we initially had planned and we kind of said we're going to spend the first week on this stuff essential Concepts and tools for Builders I want to be very clear that we are not dispensing with this we are reformatting the way we're going to cover it because what I heard over and over is people want to get into this much more quickly I Heard lots of people wanting to build with agents I Heard lots of people wanting to build QA systems lots of people saying rag to me and lots of people wanting to do more complex things so that's what we're going to jump into first we're going to have a little bit more time we're going to be covering agents as soon as this Thursday we're actually going to kick Thursday off with agents you're going to be amazed at how much you're going to get in today even and then what we're going to do is we're going to cover these Essentials through our community build sessions over the next couple of weeks and what I am going to do is I promise to keep this updated as we go so as we kind of go along here I'll keep this particular schedule up to date um if you have feedback if we're making modifications and you have feedback you feel like you didn't quite get enough of anything that was that was particularly on here um please let me know and we'll be sure to provide some additional resources we don't want to drown you in resources but we want to make sure that if you want to be underwater more in resources that we provide that for you um It's Quickly easy it's very easy to become overwhelmed with a lot of this stuff so we're going to kind of expand week two into more or less two weeks and keep week three and four the same so a little bit more on that here what we're going to do is we're going to cover finetuning next week we're going to cover instruct tuning the following week Laura and Q Laura will be part of those discussions and then we will also be covering evaluation in week four during the community session on Wednesday nights we're going to share a resources link with you right now that is also available publicly on our GitHub this is a resources link that provides a lot of different notebooks for you gives you a little bit more on fine-tuning a little bit more on some of the stuff that we've done a lot of the content we've been putting out there maybe some of you have seen it already instru tuning you know natural language to SQL fine-tuning the io schema using Bloom and a Laura approach there's lots of things that we could provide you with again we don't want to make it too overwhelming but if you're interested in this stuff tell us what interests you most and we'll continue to sharpen the tip of the spear to get you exactly what you need all right so that kind of bleeds into these Community sessions we've had a couple leading up to this particular cohort and I think they've gone well we've kind of been figuring it out together but I think we're starting to get there and last week was was very positive we have a special one this Wednesday night that's on AI video generation that's going to be run by Ali so little bit of video in here a little bit of L diffusion models if you're if you're down that Vibe instead of llms and then we're going to focus in on these fine-tuning instruct tuning and eval over the subsequent weeks Ali anything that you want to say about that upsell uh sure so uh I think there will be $15 or so that is going to be uh purchased necessary in order to go ahead and run this $10 for colab $5 for the actual script uh the script that we're using is by uh ex uh s aella uh who actually went ahead and wrote all this um and you can see right there I've generated a bunch of these videos with it um these are all with warp fusion so it's videoto video transformation uh using AI so and this is not Angelina Jolie right right this is not Angelina Jolie uh I took a different singer and I changed her into Angelina Jolie and that's what we're seeing right here if you want you can do stuff like this on Wednesday night if you if you throw down about 20 bucks um so Ali doing cool stuff uh he again he's our head of latent diffusion models he's doing this stuff all the time Chris is going to be focused on llms as you probably all already know so so yeah that's that and now we're going to get into it so uh so then Lang chain um we have a lot of content and what we don't want to do is we want to duplicate content that we've done elsewhere so for those of you that feel a little bit behind on Lang chain I'm going to show you some resources that i' encourage you to dive deeper into for those of you that feel like you're up to speed already um then just use this as a brief refresher this is an advanced course and it's meant to feel a little bit painful and a little bit hard that's why it was such a high friction process for all of you to get in so I'm not going to go through these slides but this is an event that we did do and you have access to this what I want to show you real quickly is that in order to set up a document QA system there's four primary components there's models there's prompts there's chains and then there's the indexes so I'm just going to go through this just very briefly chains are the primary abstraction the chain and Lang chain is just about putting things together and then when we talk about when we want to set up the document QA the first thing we want to set up is the model we done content on this elsewhere you are going to have to set up that chat style model this is something we provided some resources to in emails over the past couple of days but you may need to dig into to come up to speed this chat style model uses a list of chat messages as input and gets as output a single chat message this is different than using a simple llm the difference is that it's using roles so you see the three roles here we have our system system role user user Ro and assistant the assistant role is basically us acting like the AI the user is just us being us and the system is giving it some context about where it's coming from so giving it some role giving it some specific Persona giving it some place to stand rather than just on the llm model and so open Ai and Lang chain both use these terminologies you're going to have to get used to them they're similar but they're a little bit different and we're going to kind of let you figure this out as we go these give way to the prompt template that you're going to have to use and we'll see this in action here shortly and we're just essentially using the chat model context to create a prompt template so you'll see it in code shortly and once you have the chat model and The Prompt template you simply chain them together and there's your llm chain there's the chain in L chain the first length of the chain and that's kind of how we get started from there we need to then chain it to a vector store a vector store is a specific type of index and it's the index you need to get started kn as we get started with Lang chain it's okay to think of index and Vector store as synonymous but as we get into llama index you're going to want to expand your understanding of indexes to different types of indexes for now though it's okay to think of an index as a vector store so what is a vector store it's simply a place where you store a bunch of vectors that's it and those vectors each represent chunks of words taken from documents so you take the document you split it into chunks you turn those chunks into vectors through the process of embedding this is all natural language 101 you store all of those in a vector store the retriever the r in rag let's say is the thing that allows you to return a similar Vector in the store to whatever it is that you're querying whatever question it is that you're asking and that allows us to then complete the question answering chain we're not going to take questions right now because we want to push again pretty hard in this class and what we want to show you is that when you build a rag system a retrieval augmented generation system which we are calling a retrieval augmented question answering system which we'll talk about a little bit later the question is what does this thing actually look like well it looks first like this where you're taking documents you're chunking them you're sending them to let's say the open AI embedding model and then you're storing them the numbers the vectors that represent the chunks into a vector database from there you're essentially hooking this thing up to the brain to the llm and when you hook it up to the llm all of a sudden Things become much more interesting admittedly this looks more complex than it is but when we load this thing up with some Shakespeare we can ask questions like who is King leer and you can follow the logic train through and you can understand the prompt template and you can kind of connect this idea of the input in the form of a chat model message with the output in the form of a chat model message connected with this prompt template and this index or vector store and this is what we're going to have you work on right now you're going to build this yourself right now and it's going to be easier than you think and we're going to pair you up two of you at a time for 20 minutes and when you come out you can feel free to ask us whatever questions you want some of you will crush this very hard if you do we got some bonus challenges at the end for you that we'd love to hear how you did with some of you will struggle with this a little bit and that's okay we encourage you to work together in your pairs of two everyone in here is a very strong candidate and very motivated to learn this stuff we're all coming from different backgrounds and different contexts some of us are extremely powerful great coders some of us less so everyone in here can code or could code very well at one point in time so we encourage you guys to pair up learn as much as you can together we're going to share this GitHub link with you it walks you exactly through how to go into these steps you're going to need git you're going to need the terminal you're going to need your open AI key and it's all set you don't have to write any code you just have to run it if you run it all the way down to the end start writing some code making it better Chris will answer any questions you have when we come back and then we're going to take you to the next level with something a little bit more complicated and something that's a proper programming assignment so with that we're going to go ahead and straight open them up for 20 minutes and Juan you're going to hang with us this time and what we'll do is we will go around and make sure that you're okay if you look like you're in the zone we'll just pop in pop out if you need help you can raise your hand while you're in the breakout room and you can ask for help otherwise work with each other to get the help you need and have some fun guys we'll see you back in 20 minut goes fast sometimes I hope that you all had a useful time together uh we are going to have just a couple of minutes here does anybody have questions that they think would benefit the group if you're struggling You are not alone this is supposed to be a little bit challenging but Chris is available for any questions that anybody has and we would appreciate if we start getting a number of questions just put them in the chat and he'll rattle through them and if I could thumbs up thumbs down thumbs in the middle okay all right so so all right killer carpri look my keep crushing it nice inia what do you think thumbs middle thumbs down middle okay Chris you want to start hammering through some of [Music] these muted um okay well I think I think I'm not seeing anything really conceptual um or you know people are really struggling with there so if you're having sort of very tactical issues this is something that again we have a lot of very strong programmers in the cohort so if you want to go ahead and post your question in Discord or just leave it in the zoom chat we've got people that can potenti really help out and help you get kind of up to speed more quickly but what we're going to do is we will press on and we will introduce how we're going to take this to the next level first I've been told by everybody that we have to take a break because we have a two-hour session and some people will get upset so we're going to take five if you want to ask Chris questions you still can we're going to hang around we're going to take us a quick five we're going to start back at 82330 82330 keep it tight these classes are going to go quick uh so we're gonna see how we can get some Next Level llm going here and what we're going to do is I'm just going to show you another resource for you to dig into that we did in a in another event that we're happy to provide the link to but this is just a little bit about chain lit the the interface for actually building these chat Bots and what I want to show you about chain lit is that it just has a couple of really cool things that you should know about and it's got this which is pretty awesome it's called Chain of Thought reasoning and so if you're familiar with prompt engineering and Chain of Thought prompting this is something that is a really really powerful way to sort of look in and optimize your prompts I'm not sure what happening there and then it also gives you sort of a nice prompt playground to further optimize your prompt so this is all built in to a UI that you put on top of an llm called chain lit and so chain lit is something we've got a lot of content on you're going to end up learning this but I just wanted to show this briefly to you that we've got you know this previous event we did completely focused on chainlet we've got repos we've got resources you're going to see chainlet as Chris talks through this next application and this sort of Lang chain chain lit Vector stores indexes all this stuff we're going to kind of give you a bunch of stuff have you work on a bunch of stuff and can almost pattern match it together almost like case studies uh by building that's what we're really trying to play with in this particular round of curriculum and so what you're going to work on this week is you're going to work on combining open AI with Lang chain with chain lit with not chroma DB not pine cone not we V8 but in this case Facebook AI similarity Source similarity search how do we say this fast face I don't know how do you guys want to say it we say what however we want this is the state-of-the-art babbe um so Facebook's Vector store if you ask Lang chain if you're looking for something that can just run inside your node.js application in memory without other servers to stand up and go for fast so we're going to see how this works and this is your opportunity to ask questions please ask them in the chat because we are going through time very quickly Chris is going to spend about 10 minutes and then we'll give you a solid 20 minutes to get started on this assignment together in breakout room so Chris take it away show them the completed version of the assignment that they will have to complete this week all right just confirm you can see my screen with chainlet yes perfect okay so uh basically what we're going to be uh what we're going to be building today is this which is a simple uh you know retrieval augmented Q&A system use using Lang chain and chainlet uh the chainlet part is basically done for you guys so you don't have to worry too much about that but the notebook itself you'll be required to write uh so I'll just show you some examples of things that you can ask uh you know this particular QA system so we're gonna ask is Ken funny in this movie uh we get the response totally bro according to J 25's review they couldn't stop laughing Ryan Gosling's portrayal of me Ken so yeah I'd say is pretty funny this movie uh as you can see we've named the uh the actual chain Consulting the Kens all just little things you can do with chainlet uh to show you guys kind of the the ux power of chainlet we're also providing sources to some reviews so that we can verify you know what users are saying and what they're being coded as saying we want to make sure to do that so that we don't uh we know the llm will hallucinate this is a way to help guard against that by giving access to what the actual you know reviews are saying you can ask questions like uh did people like will F frell's character in this movie and we get this response where it's you know maybe maybe they didn't love him so much so uh you know we can ask all kinds of questions it's QA system and it's built fairly straightforward forwardly we're going to show you how that's done right now uh so basically what we're going to do is we're going to uh build a retrieval augmented question answering system there's some contention about the term rag uh which is I you know in terms of uh techn technically correct if we want to live in like super pedantic land uh a lot of the systems being called rag are not really using the rag technique from the paper uh but it is cloy you know it means the same thing uh as you know uh retrieval M mented question answering so the idea is we provide context which should contain the answer to our question to the llm so that I can answer the question accurately um all you'll need to do to get started the first section is just get some dependencies and put in your open AI API key uh going to be a theme in in some of these tasks and then we're going to grab some Barbie data we're going to do that using selenium and the scrapey library from python uh we could pay $50,000 IMDb for their reviews API but I think this is probably going to be a little bit easier for us uh we will have to be sure to get the chromium uh Chrome driver so that we can access Chrome inside of our collab instance once we do that we can go ahead and just load a bunch of reviews uh along with there some metadata about those reviews which we're going to store in a panda data frame just so we can easily export it out uh to a CSV file so we can use that to parse all the information we want we could use a CSV parser uh or a a pandas in memory parser but uh we're going to go through the pattern of you know having some files and and turning them into our index just so we just so we have that uh experience so we are going to use the CSV loader the CSV loader is does what it says it loads csvs uh we want to have our source column VI the review URL this is important because we will be returning sources with our uh with our queries and so have having the review URL allows us to do that in a way that makes sense so that users can just go ahead and click on the reviews uh without without you know worrying too much uh you'll notice that there are 125 reviews uh there are a thousand plus reviews on the website you don't want to scrape too much and get IMD ma IMDb mad so we're just sticking to 125 reviews uh once we have those reviews kind of loaded into documents right so the idea is that we're taking each of these CSV uh rows and then we're turning them into documents right so there is this kind of idea we I talked about this in the other notebook but there's this idea of like document is a very ambiguous term but what we're using it as here is like the NLP term which is uh you know text objects so our CSV loader converts our CSV into 125 text objects and then we're going to use use our recursive character text splitter in order to split any exceedingly long uh text contexts into smaller pieces and we want to do that because we want to only provide the most relevant information to our llm we don't want to provide a bunch of kind of extra information uh go into more detail about why that's beneficial in the other notebook but of course uh in this one it is the case that we're going to split our text a very common pattern that you'll see across the board when you're using these kinds of tools and it doesn't split it too much we only get 181 total documents out of our split text so not so bad pay attention to this particular piece because we are using uh actual documents from our loader we do want to make sure that we're using transformed documents as opposed to something like from texts next up straightforwardly we're going to create face uh vector store so this is a much better version than the version we built in the other notebook of a vector store but it's doing the same thing at the end of the day which is just storing tons of vectors right so um this is the idea of a vector store and this is what we're going to become familiar with throughout the course of this uh program and we are GNA right away we're going to throw a little bit of optimization at you guys just to uh just so you can see some of the patterns we're going to be looking at later on which is we're going to use a cash backed embeddings uh so instead of the standard embeddings model which doesn't care what else it's C uh it's embedded we're going to use this cash back embedding in order to you know have a a cache of what we've already embedded so if we get the same query multiple times we're going to get you know a faster inference the second time because we won't have to embed that query we already know kind of what's what's happening there uh recursive splitter works better than sent splitter it can it depends on the kind of document you're using so if you're using documents that have kind of these like very familiar structured patterns then um you know this the recursive splitter is actually quite good uh if you're not then sentence splitter can be very good especially if you understand your data well and you know that you're going to get a lot of context in sentences and maybe uh you know you don't need so much outside of sentences that's like one of those keys you've got to know your data um but we build our catback embeddings and then we built our or we build our face Vector store and now we can query it so you know as talked about the other notebook this idea is that we've stored a bunch of vectors in this uh Vector store and now we're going to embed our query with the same model to get a vector and we're going to see what document vectors are closest to our query vector and return those as you can see we're returning four documents just as an example and we get this idea of uh the the response so this is what we're going to get in our response everything including the review URL awesome so very happy about that and there you go we're going to uh just showcase very briefly that this cache thing works uh so runit Euro C lab and you'll see the the results it's great um and then of course we are going to use return document our source documents equal to true because LMS love to hallucinate and there is some evidence that you know these retrieval augmentation methods actually make those hallucinations better uh meaning that they are they are less egregious let's say uh we still want to just give our users the ultimate control and provide them sources so they can come to their own conclusions uh we're going to to set up our our chain this is a a pattern that you'll become very familiar with if you stay in line chain ecosystem which is we set up an LM we set up a retriever and then we create a chain that uh uses both of those in order to uh give us exactly what we want which is we want an llm to parse results from our retrieved documents and we want a retriever to be able to retrieve documents and there we go we're also using a callback the main reason we're using the Callback is just to introduce the idea of callbacks we'll be using them more extensively as we move through the course but for right now it's just the idea that we have this thing that we can do that lets us interact with our chain while it's happening in between steps and so we're using this standard out uh call back Handler which is just going to print everything that it's thinking out to our standard out which is just uh the The Notebook output cell um so that's all we have to do there and then we can start using it like you'd expect so so how is well farell in this movie uh we get to see all of the source documents as well as the result uh and then we can ask do reviewers consider this movie Ken enough which is if you're from familiar with the movie uh you know this is a they do this all the time with the using the Ken so um and they uh they they you know the answer is good basically so some people think yes it is K off some people think it could be better so uh but that is it that's the idea so you'll see this is a completed version of The Notebook the version that you guys are going to see in the actual um repository that you're working out of is not going to be completed so it's going to be up to you to uh make some of these decisions and write some of this code uh in order for it to you know work as intended so uh you don't have to write the Parson code or the scraping code I thought that was probably a little a little cruel so uh that that will be that will be handled for you guys uh but that is it so all right that's uh that's K right there uh Chris I looks like we have one question from inia just curious if recursive splitter works better than pented splitter yeah I answered that during the uh demo so yeah great uh okay so we have about 15 minutes and we want to send you to breakout rooms in a little bit larger rooms and meet a couple other people and maybe it's going to be most interesting to talk about your strategy for completing this assignment uh and getting this thing to work and if you know you guys are chatting and you come up with anything you want to share it with the larger group hey we're getting together on dayx at whatever time you know let the larger group know let everybody know um because I'm sure this is going to be challenging for quite a lot of folks and it's supposed to be very much designed to be we will release the completed version of The Notebook on Friday night EST after Chris's office hours he's going to go ahead and release the answers in case you want to just wait till the weekend look at the answers and just submit that you will be able to but with that we're going to go ahead and keep the time moving along here and we've got 15 minute breakout rooms that we're going to open up keep the questions coming in Discord raise your hand in the breakout room and we will do our best to help you out we know this is a lot um welcome to the party all right Ali let's open up all right welcome back welcome back welcome back we are at 8:55 pm. eastern time I hope everybody enjoyed the overview the discussion um does anybody have any shout outs of of things that they're planning to to do to maybe get this thing completed any Community Vibes type type things they want to shout out real quick maybe one minute everybody is completely posted perfect yes okay as designed okay so what do we learned today rag is basically Rocka we can be all technocrat pet about that and we should be and you should discuss it with us in office hours question answering if you learn how to do this you are absolutely dynamically dangerous in every company on the planet and you should be paid more okay that's why you need to go get after this once you know the basics you're going to be able to just Crush any application that's the fuel that you need to get down to business

Channel: AI Makerspace

Views: 3,079

Keywords:

Id: d1Oj5vrTWC4

Length: 55min 49sec (3349 seconds)

Published: Tue Feb 20 2024