Retrieval Augmented Generation with OpenAI/GPT and Chroma

retrieval augmented generation or rag is a technique for enhancing large language models by providing extra data to help them answer questions in this video we're going to learn how to use this technique to answer questions about the Wimbledon 2023 tennis tournament using open Ai and the chroma DB Vector database let's open up a Jupiter notebook we're going to start by downloading the Wimbledon 2023 Wikipedia page using langchain a framework for developing applications powered by large language models so let's start by importing some modules from there so we're going to get the Wikipedia loader and we're also going to get the recursive character text splitter so we'll use that in a minute we're going to use the Wikipedia loader to get the data for the Wimbledon 2023 page which has tournament information results and a bunch of other stuff as well so let's create a search term so the search term to find this page is 2023 Wimbledon championships and then we'll use the Wikipedia loader we'll pass in the search term and we're going to tell it just get me one document now you can use this loader to get as many documents as you want but we just want the one and once we've done that we're going to split the text up using that recursive character text splitter we're going to go into chunks of 100 characters and we'll have a little bit of overlap and it's basically going to kind of start from the paragraph and sort of go down and try and break it up into into chunks now the reason to do that is because we only want to provide relevant information to the llm later on rather than just throwing everything at it once we've done that we can then let's have a look at three of the the documents and you can see we get a document it has like a bit of the the text from from the Wikipedia page and you there are actually a lot more of those what we want to do next is actually go and store those chunks of text and we can say we're going to store them as embeddings in chroma DB and again there are modules in Lang chain that we can import so we're going to import the one for Chroma and we're going to import another one for the open AI embeddings let's initialize the open Ai embeddings and next we're going to store that Wikipedia text those documents in chroma DB which calls itself the AI native open source embedding database now in a nutshell that means it lets you store chunks of text as arrays of numbers or embeddings and then do nearest neighbor search on those embeddings to find the the most relevant answer to the the question that you've asked so let's create a store so we're going to go chroma from documents and we'll pass in those those documents that we created and we're going to pass in the open AI embedding model as well and then we'll create some ideas and a collection name and we'll persist it now internally what's going to happen is chroma is going to convert those text chunks those chunks of those documents into vectors numbers using the open AI embeddings model so that's all hidden from us and then it gives us back the other store now let's have a look at what sort of questions uh we can ask so again we're going to just import some modules from langjang and we're going to create what we need to do next is we need to create a template uh and then we're going to say we're going to create a templates we'll tell it hey you're the Wimbledon 23 2023 but and we'll also tell it if you don't know the answer the answer to the question is not there don't don't try and make it up and then there are some built-in parameters so we've got context and we've got questions so context is basically where it's going to put in the the data that it uh extracts the extra data that it's giving to the model and then the question will be whatever question we answer and then we'll create our prompt template let's initialize the open AI model we'll give it a temperature of zero which kind of means uh it shouldn't uh it should be you should get reasonably nearly the same answer each time you use it rather than having variability and we'll use gpt4 but I mean we could try one of the others as well next we're going to create a question and answer model in Lang chain and we're going to say we're going to pass in the llm we're going to pass in we need to make sure we pass in the the chroma database as a retriever so that tells it hey this is what you're going to use to look up the relevant documents and then that extra cool thing that you can do with this is you can say I want you to return the source document so when you get the answer you can have a look what documents did you use to answer the question right now that we've done that let's use that QA with source to ask you a question so we'll start with something that was at the top of the page so where and when was Wimbledon 2023 held and we'll give that a few seconds so take each of these takes three to four seconds or a bit longer and you can see it comes back and it says one button 2023 was held at the all England lawn tennis and croquet Club in Wimbledon uh from the third to the 16th of July 2023 so that's a good start how about we ask it who won the men's singles title and what was the score so we can ask that give it a few seconds it says Carlos won the title and then if you and that gives you the score as well and if you look underneath it actually has picked out the the correct part of the Wikipedia page and I expect if we'd asked it who did who did Carlos beat it would be able to tell us an effect Djokovic and you can see that that information is there as well how about another one so in 2022 Wimbledon uh sort of stood on its own for Banning Russian and belarussian uh tennis players because of the uh the war in Ukraine um and so this year they actually allowed them back so let's see if it can pick that up so I'll ask it where Russian players allowed to play and you can see it says it doesn't know says the context provided uh doesn't uh does not specify whether the Russian players were allowed to play but weirdly if you look just underneath that it does have the the chunk that says that they are allowed to play so I thought okay let's let's try asking it in a different in a different way so did Russian players play and this time it says yes we saw the return of Russian and Bella Russian tennis players I'm not entirely sure why that is that's kind of kind of a bit a bit quirky but if anybody knows let me know let me know in the comments why would we get a different answer asking the question in a slightly different way I thought let's ask about British players it doesn't actually have anything any information about British players in there and that's exactly uh what it tells us I suppose you would want it to say like actually if I don't have any information about them I can probably assume that they are allowed to play but maybe we would need to add more uh context uh or more information to our prompt so it could do that uh one more question so were there any extra events held and so yeah and it identifies yes there was there was a special event where Roger Federer was honored uh if you liked this video you might like this other one as well showing how to run a hugging face large language model on your own machine

Channel: Learn Data with Mark

Views: 8,077

Keywords:

Id: Cim1lNXvCzY

Length: 5min 44sec (344 seconds)

Published: Fri Sep 01 2023