Build a Healthcare Search Tool using Mixtral 8x7B LLM and Haystack

hello everyone welcome to AI anytime channel in this video we are going to develop a question answering QA tool for biomed literature search now we're going to leverage Pub made uh as a data source Pub made is a free search engine for biomedical literatures very famous you know if you are looking to query some recent case studies some kind of you know information that is already available on PubMed you know for something related to healthcare mainly you can get it from PubMed now we're going to use that as a data source and llm will act as an augmentation layer so we'll have an llm augmented question answering tool where you can you know connect with Pub made and you know in retrieve information from that using a large language model so as a large language model we'll use Mixr model okay so Mixr aex 7B which which is a mixture of experts M Moe modeled by a French startup named Mistral AI now mistal ai has created this Moes model nowes are you know getting famous uh uh lately because know people have started creating different mes mixture of experts where these models are working in tandem having different expertise or specializations now when you talk about atex 7B you know uh it's important because there are those models which are working together and they all have different expertise so depending on what query you are asking one of the model gets triggered and you know you it it works for whatever uh ug case or task that you are executing for now in this video we're going to build uh using an orchestration framework called Hy stack okay so thanks to Hy stack team for making this notebook publicly available where they also have a pub made feature so you can connect with the pub made directly and get the information and you can use a large language model to augment those information to retrieve a humanik response so basically a perplexity like a tool that we're going to build but that for pubid search engine so let me show you quickly that what we are going to build in this video so if you look at over here and let me wear my glasses uh if you look at here it says llm augmented QA over Pub made search engine and you know it's a gradu application it's a gradio UI that we have built and you feel free to build any other UI as you want because the code base will be available on my GitHub repository now if you look at here I have some examples of different questions related to you know Healthcare medical and whatnot now if you see here one is how are mRNA vaccines being used for cancer treatment so once I run this it start processing you can see on the right hand side it says processing you know takes around 7 to 8 second now it's almost you know it's going a bit up because it acts as a you can now around 12 seconds we got our response it says the use of mRNA vaccines for cancer treatment is a promising area of research now if you are working as a research you know a research fellow or if you're working as a uh in a hospital or or in a uh a healthcare institution where you have to retrieve information have to go through different case studies do a lot of research so you can use pubid uh people use pubit for that but how you can use a large language model that augments that search that you use now this is what we are doing here now these vaccines work by introducing mRNA into the body which instruct cell to produce specific proteins that can trigger an immune response and blah blah blah right it gives you the answer in very detailed manner over here now you can do a lot of post post processing which I haven't done here for example if you look at it does not stop so you can do a post post processing to handle those things but this is what we are going to build let me make it up bigger so you can read it as well now if you see the this is the output that we have got uh by asking this question now let me just clear this now once I clear this I'll make it uh let me ask the other questions I'm going to ask suggest me some case studies related to pneumonia now I'm asking this question what it does goes to permid fetches the relevant information and I'll show you that how it might be fetching it once we write the code you know through different keywords and you know the promt template that we will Define it connect with the pubit and we'll use a pubit feature by HT uh Library which is an orchestration framework Alternatives of Lang chain Lama index semantic kernel and some of the others as well now hey Tech have created this permit feature based on your query it goes to permit FES the uh you know retrieved uh information paragraphs context similar paragraphs whatever you call it a pilora of information and then that pass to a large language model that basically synthesize the response to retrieve the humanlike response that you see currently on the screen you can see it says it gives you a case study I've asked for case study related to pneumonia and it says the study aimed to investigate the association between the use of antibiotics in the first year of of the life and the risk of developing asema in childhood and it gives you a detail uh information about that so perfect now this is what we're going to build uh in this video guys to to start building this me just submit now to start building this you need a few things you need uh hug face API because you can also maybe download locally and use as if you have a compute power but I'm going to use the M model which requires High compute so I'm going to use hugging F token uh from my hugging F repository and I'm going to use the llm from hugging face directly now that's the one thing that you need the other thing that you need need is to have to install a lot of librar so let's let's see for that answer and then we'll go and start developing this application uh locally so let's wait for this and see what it responds and you can see it has given you the answer significant morbidity and mortality worldwide blah blah blah and it has given you that how you can first identify how you can treat as well RT lamp has been sown to a sensitive of whatever diagnostic station a lot of things that is happening so basically it basically augments you in your research and Discovery if you are trying to retrieve anything from the permit search engine so let's start developing this application guys all right so to develop this application I'm going to use VSS code and you can see that I have a folder and I'm going to open in terminal and activate my environment so I already have a virtual environment I'll show you what are the things that you need to install and then we'll be set to work with this now let me just do couple of City dot dot clean if you don't want to watch the video you can directly go to the code but I will recommend you watch the video where I can give you some tips as well when I'm writing the code now uh Healthcare chatbot let's open that in vs code I'll minimize this thingy now we're going to use H tack guys so to use H tack you can see this is H tack by Deep I'm a huge fan of Hy stack and the entire Community they have done a great job and they also have come up with the new library called Hy stack AI okay which is like Hy stack 2.0 now I have used Hy stack when it was like pretty old you know once they started they have they had extractive you know summarization abstractive extractive q and what not where we uh you know before Chad GPT era you can see that where we were used to rely on the B based model so I have a huge fan of hch now hch is like a very competitive you know alternative of Lang chain because they say that they are production ready so if you want to develop a new skill I will recommend you that you can you know develop your skills in the area of H stack you can work with the Hy stack and you can do the pretty much same thing that you do with Lang chain and and Lama index now choice is yours now so here the H tack is I'll give the link in description you can go through their documentations tutorials and they have a great Community as well you know and most of my codes will be from their official blogs okay but I also have made some changes and the application that we're going to build so now let's go uh let's jump in and uh start writing the code so I'm going to create an app.py and you need an hugging face token so if I can show you I have these numbers here that you see there are four tokens that I already have you can have your own tokens of course and you know you have read and write access when it comes to access tokens on hugging face write is something when you push a data set or you push a model to hugging face Hub that's why you need a write but for downloading the model you just only need the read access so just get it from there now the first thing I'm going to do is import I already have imported but before import let me show you these are the things that you need you need Hy St AI because that's their uh new library then need then you need piade so you have piade then you need gradio which is which will help you develop the web UI and then in the EnV because I have saved my secrets in EnV and then you need Transformers now let's come here and let me show you I'll go to my you know other gist as well so let me just open my your gist because I want to save some time so I'll just take some of the inputs from here I already have created handy and let's come back and paste it here now if a look at what we are doing we are saying from pade import the pub made so the pub made is available in that particular library now we are using H stack the components the document I'll explain that once we reach there now components generator we are hugg we're going to use hugging face tzi generator that within that we're going to use the Mixr Mo model x7b through tzi text generation interface if you don't know that so text generation that we're going to use use from hugging face and then we have pipeline from hugging face because H Tech excuse me sorry the way you use chains in Lang chain here you use Pipeline and I feel that's a better way in Productions you know you you can look at the way that H stack provides through a you know a pipeline where you run things sequentially you can also do that now that's the beauty of H tack now you have prompt Builder that helps you with the prompt you have prompt template in Lang chain so very much similar what we do with Lang chain as well now what I'm going to do is load. EnV because we going to load our EnV key so let's do os. inv okay so os. inan and the way you define so I have defined hugging phore aior key and then excuse me I don't know why it became small API key and then just do os. getv so let me just do .g EnV so .g EnV and then you just pass this in parenthesis you are set to load your API key now you can also use the same code base locally as well you know you have to download the model files from the uh hugging face repository wherever you use the model you can also use the block model for that you can use a python Lama CPP or you can also use C Transformer now the choice is yours if you have the compute power please go ahead and do that now here just for the capability I've been using a directly from hugging face now os. invine is done the next thing is we're going to connect with the pubit so let me just do that so what I'm going to do is pubid equals and I'm going to use pmid class that we have here and we also imported there now it takes couple of params one is tool the other is email now tool is important and email can be a dummy email so let's see that so I'm going to write tool and here it's a tool where it say string so you can to give that string so I'm going to write htag that's what they suggest they say Okay H tag 2. no o and then you write Pro protype so you can write that H 2.0 prototype basically a string value and now in the email you can also give a dummy email so let me just write dummy email at the rate gmail.com now I have my pubit ready with uh this now I'm going to write a function for document so let me just get the doc we're going to do a documenation so let me just do Define docu meti and here I'm going to pass my article so let me just pass the article here and in article I'm going to write here in article let's do return document and that's why we have imported document on line number four if you see and here I'm going to write content equals article dot I'm going to get the abstract from this article so you're going to write abstract you can see it's right abstract and then a metadata kind of a Things So Meta The Meta tax now for the meta tax I'm going to have a title so let's it's a key value pairer so let me just write title and then you title write article. title and then you write keywords article. keywords fantastic we are done with our first function guys which is defi Define documenti now I'm going to create a component so in h stack these are terminologies that you have to learn if you want to learn more I will recommend that watch my previous video have lot of videos on H tag you know different rag pipelines in h tag I created a video when there was no Library like hey stack AI you know I created a video where using llama 2 I remember that and there was no support officially by htac but I did it through uh some of the custom based models they have you know node modules any stuff so you can watch that video I'll give the link in description now Define documenti article now the next is create a component so it's like a you define like a decorator so let me just do a component here and now I'm going to write write the pub made feature so let me just do Pub and this is where you will write Pub made fature so let me just uh do that and let me see if I have let me go back to all my gist I like to see if I have that class just to save some time otherwise I have to write that okay it seems like I don't have but it's okay now let's come back here now Pub made feature the first thing that I'm going to do is I'm going to define the output type so I need a so how I can do that is component. output you can see this is right output times excuse me I'm so sorry output types output types and then passing the Articles and articles equals and then going write list and inside list I'm going to pass document now this is how you define but it's a list so this should not be excuse me this is wrong now list and document now this is the Articles list document now once you have that let's write a function called run so Define run and inside this function I'm going to pass self queries once you have self queries let's pass list string so let me just do that self queries excuse me and now inside this function I'm going to write the clean query so let me just do a cleancore query of variable now clean queries equals queries and then want to split that so split not split strip strip clean query strip and then I'm going to strip that and then split with the line break so strip is a function so strip and then I'm going to do split this is correct split this is now okay now we have this I'm going to define the article which going to be an empty list so let me just get the Articles empty list over here and we're going to append that l so let's have a try cash so if try works it's fine otherwise the query is not able to get any kind of you know article from pmit so let's have a try catch not try catch I'm talking about Java here try except try and for query in cleaned queries and if the query is cleaned then have a response from pmit so response equals permit. query and permit has a query function that you can see inside this query I'm going to write query and then Max result so max result equals 1 now this is what I'm going to write in response now we have the response so let's have a document so documents equals now in the documents I'm going to write documenti the function and that in function I'm going to pass my article so article for article in response this is not respond or document only response and this should work documenti article for article in response and then articles do extend so let me just use the extend function and pass the documents we are okay with this try now let's have an exception so except yes correct I think it was right that was showing me except exeption as e and then I'm going to print the error here so let's print error and M the exception and then I'm going to write here manually so F couldn't could not Fage articles for let's write queries then because we have queries and and then just pass the queries here and I think we are okay now accept is done and just get the result so the try and the accept and then result so results equals you're going to pass your it's a key value so let's have your all of your articles articles and then just return the result and we are okay with our permit feature component so this is the component that we have created guys a permit of fature component now we're going to use that later let me just this is surprising I will just return result I will just close the vs code and I will start again code DOT fine and then we write result equals and then in the result I'm going to write articles and then you write articles equals articles and then you just WR going to write return result now this function is done the this function is ready we have a component that we are working with now I'm going to have a keyword llm so keyword llm and going to use hugging face TGI generator now inside this you going to pass the mistal AI so let's come here and show you now Mistral AI hugging face if you write this come here on Mistral okay now uh you can open you can see they have lot of five models over here and atex 7B instruct this is the model that we need so M has five model they also have released proprietary model so they also are going commercial so there's nothing wrong in that now if you copy find the keyword llm this is the model that we're going to use okay in this video atex 7B instruct models which is further fine tune on the instruction data set on the pre-rain fine TR model let's have and wrap uh warmup so it's it has a warm-up function so I'm going to use warmup function now this looks nice now the next is llm this is for keyword again then I'm going to Define find it same and let's copy this entire thing so I'll tell you why we are doing it twice one is for keyboard maybe for keyword you can use a smaller model but I'm going to rely on the same one you can also use keyboard to extract keyboard from the Articles that's also be a good approach I just thought of it llm warm up now we need two prompts let me just show the prompts here so let me just come back on my gist and let me come back here on the prompts go to raw earlier we have seen this and now I'm going to paste it here now once you paste this it shows you keyword prom template and ALT G so you can read it if you read this it says your task is to convert the following questions into three keywords so when you ask a question it basically get the three keyword it will help you search permid the keyword based search on permid three keyword that can be used to find relevant medical research papers on permit here's an example what are the latest treatment for major depressive disorder so the keywords can be you know uh anti-depressive agent depressive disorder major treatment blah blah blah and this is basically has been created by the large language model because you need that capability to you know infer the key uh query and then create three keywords that's what we are doing here and the question and then this prompt Returns the keyword now the next is our main prompt template that goes to llm for the output it says answer the question so you can read that answer the question excuse me I'll just see my monitor uh okay it says it answer the question truthfully based on given documents if the questions don't contain an answer use your existing knowledge base so if it's not available on Pub made so the model still has you know ATX 7B it has their own uh Aon vectors right is their own knowledge base contains in different vectors so it has your own knowledge base that you can retrieve the information from so it pass the questions then you have your articles and in the Articles what we are saying for article in articles article content keyword title meta because we have defined that if you look at on the document eyes as well the title The keywords and whatnot so these are the two prompt that we have the next thing that we're going to do is create a keyword prompt Builder so we're going to use the prompt Builder so for the first let's do keyword prompt unor Builder thingy over here so Builder equals and I'm going to write prompt Builder and inside the prompt Builder I'm going to pass my template so template equals what was that prompt template or whatever we are done with keyword prompt Builder same goes for our prompt Builder which is for llm so let's write prompt Builder equals and then again the same prompt Builder and this time it will not be qord prompt Builder it will be only template equals to prompt template so let me just write prom template and we have our prompt Builder now now once you have the prom Builder let's use the fure so I'm going to write feure a variable and then I'm going to use the fer here pubid fure component so let's use the pubid feure now we get the feature here now once you have the feure let's create a pipeline so what happens is you use a pipeline keep adding components in those pipeline so basically it's very scalable as well you can use H tack to scale your llm powered application in production as well because it follows a very traditional way of where we used to work with deep learning and machine learning uh you know our systems earlier you know through a very scalable approach through pipelines now I'm going to use pipeline here so pipe equals pipeline so let me just write Pipeline and it's a class so I'm just calling the pipeline class now let me just call this now I'm going to keep on adding components first it also has component also has node connectors connectors and M node so let's see the first is pipe dot add component and you can see the first thing that is you can also there are there are two things you can do you can only just write if you can also give a bit of you know naming documentation you can just name that component so it will be easier for other people who are reading your code so for example add component I'm going to write here keyword prompt Builder so you can just write that and you can keyword prompt Builder and then you can just write keyword prompt Builder so this is how you define a component so pipe. add component and I'm going to write keyword llm so keyword llm and then you can just write keyword llm so let's write that then I'm going to write add component and here in add component I'm going to write permit the fature so once you give that the next thing is that your fure will be completely in a sequential order that you see first is that you put it it basically use gets the keyword then is the get the pubit feature and then it gives the prompt builder in the prompt template and then it executes the llm for the answer that's what we are doing so pubit feature I'm going to write feure here so let's do fure and then again pipe. add component and the next thing is your uh prompt Builder so let's just write prompt underscore excuse me prompt underscore Builder and and then you define your prompt Builder over here so prompt Builder and then what next uh llm so pipe. add component and then you write llm llm now this is what it is now let's do pipe. connect so I'm going to write connect and the connection is how do we connect first so let's see we have our keyword uncore prompt Builder so keyword uncore prompt Builder and then the prompt so do prompt and then you connect this with the llm prompt so let me just write keyword uncore llm do prompt that's the first connect now pipe. connect and then I'm going to write keyword llm they get the replies so let's just get keyword uncore llm do replies and then this with LM repes keyword llm that is done no the next would be pmit feature so the next would be pmit feature. query so pmit under _ fure and I hope I am spelling it right queries otherwise it will throw an error pipe. connect the next is pipe. connect and inside this I'm going to write pmit will fetch the article so pmit uncore fetch. articles and then this goes with prompt uncore builder then once you have the article you pass that in the Builder prompt Builder with the Articles so articles and and then you just again do pipe. connect and then here you just do prompt builder. prompt prompt builder. prompt and then this pass with llm prompt and then you will get your output llm prompt and we are done with our connection now let's write a function that will basically gives you let me see if I have that function I'm not sure I have to go back so we have the input we have the questions we have the prompt we don't have that okay now let's write that so I'm going to write a function here called ask so in this function I'm going to pass my question and you pass a question here so the outputs would be outputs would be pipe do run and inside this run I'm going to write data equals and you write keyword uncore prompt uncore Builder prompt uncore Builder and here you define that [Music] question question and then you define question here so this is what it goes output question we have keyword from Builder question now this should be now the next should be prompt Builder so let's just do a prompt uncore Builder and in prompt Builder I'm going to again pass my question and then question and then goes your question again now this is cool now the next thing is llm so llm and llm also again uh key value pair which is Generation quars you can just Define a lot of quars here you know tokens and what not temperature stuffs now I'm going to Define generation qua and then inside this again you going to pass I'm going to write to Max new tokens as one if you're using open AI you just have to do uh Max tokens not new word okay and then you going to keep let's keep it 500 and then I think we are done with this now this is my output variable that gives me the output so let's probably let's print that okay so print question and then print output so if you print output it will give you a raw content so I know how you can pass that so llm and then replies and then the first one so let's just also return that so return and I'm just going to copy it from here to here if we are want to call the function ah excuse me also track in terminal that's why we have printed it to see if that works otherwise it might give some error now once you write that the next thing that we want to do is the gradio thingy so let's write the gradio thing now for the gradio it's easy so I'm going to use the gradio chatbot interface so I'm going to write I face and we have gr from gradio and I'm going to use the interface so you can see it has an interface come down it will see a class interface I'm just going to use that because it's easier to create a chatboard with this which takes a function so FN equals ask because that's our function name and then inputs so my inputs are nothing but a text box so let's define a text box so GR do textbox uh you can also give a text area but I'm going to give text box again inside the text box you can do a value like a placeholder so value equals and I'm going to write like for example uh how are mRNA vaccines are or like vaccines being used for the Cancer Treatments now this is my question as a like it's a value it's a placeholder now once you write the output the next thing we're going to do is what is should be your output so I'm going to keep mark down as my outputs so let's have a format defined for that so markdown outputs mark down and excuse me and then you give title of that okay so title should be uh llm augmented LM augmented uh Q and A over permit search engine something like this now you have your title let's give a description so I'm going to give description equals and in description I'm going to write ask a question about the about biomedical literature now I think this is fine you can customize it the code will be available now you're going to give examples I already have created some examples just to save some time I want to write a lot of things now these are the questions of examples let's go raw contr a contrl c come over here paste that I'm going to have my examples and hopefully this is correct okay let me just come back and okay cool now we have that done now I'm going to have my theme so theme equals and let's use the soft theme so grad has base default monochrome color soft lot of different types of themes that you can use it so I'm going to use gr dot uh themes uh do sofd I'm going to use a soft theme so it's a class that you can see it's a soft that we use and it gives you the color like blue and all some soft colors and stuff now I'm going to use allow flagging so I'm going to keep allow flagging equals never for now okay and we are okay I think now let's launch that so if face. launch and you can if you are you can also do s equals to true if you want to get a link I'm just going to do debug equals true so debug equals true if I'm going getting any error now we are okay with our code guys this is what we did we built a quickly an app using a pubit htag pubit Mixr 87b and that's what we use uh to build this now let's run this and see you know uh if this works let me open Terminal and run python app.py once I do python app.py I'm expecting that it will open a gradio app you can see it says running on local URL 786 KN now let me open this in a new window and I'm going to just copy maybe just copy this here here so let's copy this come here paste and you can see let me make it a bit smaller because last time I made it like you can see this is what we have built up llm augmented QA over permit search engine that's what I shown you in the beginning also now you going to ask a lot of questions I'm asking about you know tell me about HIV aids for example you know I'm not a guy who have domain expertise in this field but you know maybe if you are a domain expert in you know biomedical literature if you are a doctor or you are a medical practitioner or a research fellow you will be something that you need this kind of system or tool to work with where you have to retrieve lot of information discover a lot of new information which is available on pubit you know different other sources like pubid the Google Scholar and whatnot right so maybe this will help you now you can see it says HIV remains a leading uh cause of death for adolesence and young people age 10 to 24 years HIV prevention requires multi sectoral approaches which is fantastic I loved it Target adolesence and young people addressing HIV interventions combine economic strengthening Health capabilities and gender equality education best practices are unknown because evidence on multis sector programming targeted so you got some response which is fantastic a very detailed response because we have a Max new tokens as 500 now this is one question let me ask about some other questions on autoimmune disorders now I'm asking one more question just to show you that how it works I if probably you can improve it further because uh maybe this can be improved further I feel through better keyword extractions might be right not right now it's a very naive way of you know extracting keyword and passing as a prompt Builder but maybe that's something on you if you are playing with it now but this works fine you can see it gives you a case study I think it's case report of a patient who with I because I asked a case study if you look at the question let me make it a bit bigger if you look at I suggest some case studies related to autoimmune disorders and it gives you it says the article describes a case report it gives you a case studies with a rare autoimmune disorder called igg4 related disease the patient was a 62 years old man who presented with chronic sclerosing you know it's very difficult to pronunciate I don't know how to pronunciate this medical terminologies but something called dcro adenitis or something adentis is a condition condition characterized by swelling of the lacrimal gland I can pronunciate this one condition characterized by some area in lacrimal gland where there's a swelling there's basically an autoimmune disorder called igg4 the patient went under a biopsy which confirm the diagonosis you with a combination of corticosteroids and wow this is fantastic right it gives you a hint because if you are struggling to see how you can at least get started to uh start a treatment of your patient on this kind of disease you can get some previous case studies or recent case studies so this is what I wanted to build guys you know in this video thanks to the creators of htac uh for doing all the amazing job please give and I'll give all the credits to them and if you want to learn htag please follow the documentations blogs the community the GitHub of htag and you will get a lot of tutorials articles and blogs uh on H to work with this cool applications the code will be available on my GI Repository just go ahead Fu that or download it and see if that works in your machine if you have any thoughts questions or feedbacks please let me know in the comment box more than happy to help you can also reach out to me through my social media channel find those information on channels banner and channels about us if you like the video please hit the like icon and if you haven't subscribed the channel yet please do subscribe the channel and share the video and Channel with your friends and to peer thank you so much for watching see you in the next one

Channel: AI Anytime

Views: 3,587

Keywords: ai anytime, AI Anytime, generative ai, gen ai, LLM, RAG, AI chatbot, chatbots, python, openai, tech, coding, machine learning, ML, NLP, deep learning, computer vision, chatgpt, gemini, google, meta ai, langchain, llama index, vector database, build healthcare chatbot, healthcare, healthcare chatbot, how to build a chatbot, medical, pubmed, PubMed, 8x7b, 8x7B, 8x7, Mistral AI, Mistral, mistral, RAG chatbot, biomedical literature, llama 2, gpt4v, multimodal, multimodal RAG, perplexity, doctor, search

Id: WhSbuGEbAR0

Length: 38min 31sec (2311 seconds)

Published: Mon Jan 22 2024