CrewAI RAG: How I Created AI Assistants to Run My News Agency

this is amazing now we have rag for crew AI in this we are going to create new search agent writer agent and then they are going to give us a report the main difference comes on how they retrieve data and save it in a vector database consider a news agency the news search agent search for news articles and then give the top news to the writer the writer search for more information on a topic by searching the internet to give in-depth report on each topic that's exactly what we're going to replicate here so as a news search agent going to search the news using API let's say he's going to find news from BBC website and then going to store all the information in chroma database as embeddings next the new search agent pause the news information to the writer now the writer can verify the information from chroma DB then the writer can further search the internet for each individual topic to get more information and finally we get a report like this so I asked the agent to search for latest AI news articles and I got this that's exactly what we're going to see today let's get [Music] started hi everyone I'm really excited to show you about rag that isable augmented generation for crew AI so the way we are going to implement this is using tools in crew AI I'm going to take you through step by step on how to build this but before that I regularly create videos in requests to Artificial Intelligence on my YouTube channel so do subscribe and click the Bell icon to stay tuned make sure you click the like button so this video can be helpful for many others like you first P install crew AI Lang chain Community Lang chain open AI request Duck Duck Go search and chroma DB and then click enter now export your news API key like this and then click enter you can get news API key from newsapi.org website and you have fre here as well this is used to get the latest news articles next export your open AI API key like this and then click enter now let's create a file called app.py and then let's open it first from crew aai import agent TOS crew and process next chat open AI base retriever open AI embeddings tool web base loader request and Os recursive character text splitter chroma dug du go search run so we're going to use web base loaders used to load the content from a URL the content will be split into chunks using recursive character text splitter then it's converted to embeddings using open AI embeddings then it's stored in chroma DB first we are going to Define our embedding function then defining the lodge language model totally we going to create three tools one tool to extract the data from news API and save it in chroma DB next tool to retrieve the data and give it to the writer and the third tool is to search the internet using Doug Duo search so tool number one save news article in a database so this is going to search the news article using newsapi.org API we are going to search for the top five results sorted by the publish date next we are retrieving those top five articles using request.get next we are checking if it worked or not if it fails it will return fail to retrieve news next we are going to get the Articles from the response now we are defining all splits now we are looping through those articles that the top five articles first step we are reading the content from the page using web based loader next we are dividing that content into chunks using recursive character text splitter next we are loading the divided content into all splits variable now now we're going to index the accumulated content so we are using chroma DB and we are passing all the data in all splits then we are defining the embedding function that is embedding then we are defining the persist directory that's where a folder will get created and all your embeddings or the data will get saved this can be used later to retrieve now we're going to use Vector store similarity search and then retrieve the Articles from the vector database finally we returning the data so overall the search news DB tool will go through the top five search results convert those to embeddings and save that in chroma DB and return the results now the second tool is to retrieve the store data from chroma DB so we are mentioning the path of chroma DB then based on the query or the question we going to search the chroma DB and return the results third we are going to search for news articles on on the web we are going to use Duck Duck Go Search tool now we going to create these two agents and then assign toss to them so number two creating agents first the new search agent this agent is going to search the latest news from the API tool and then save that in Vector database finally it's going to return key points from the latest news article the next agent wrer agent so the wrer agent is expert in crafting engaging narratives from complex information so the writer agent is going to use get news tool that's retrieving the save data that's how the rag comes into place so retrieving data from embeddings and use that to give us the result it is also going to use Search tool to search each individual topic next we're going to Define two different tasks for those agents so the new search task is to search for AI 2024 you can replace this keyword with anything you want to search search and create key points for each news and we are assigning that to the news search agent next write a task go step by step identify all the topics received use the get news tool to verify each topic by going through one by one use the search tool to search for information on each topic one by one write in-depth summary for the information received assign to the wrer agent next the final step creating the crew new crew equals crew we are assigning those agents and the toss process is sequential you can even try hierarchial and test the results now result equals new screw. kickoff and finally we're going to print the results that's it as a quick overview we created three different tools first tool to search for news and save it in the database second retrieve those saved news from the database third to search the web next we create created agents then we created task for those agents finally we created the crew and made them work together now I'm going to run this code in your terminal Python app.py and then click enter now I can see it's executing and you can see first it's going to the news search agent and it's using news DB tool so it's going to use the news API and get the top five news in regards to AI then it's going to finalize the list of points collected here now then it s into the writer agent so the writer agent is using get news tool to verify the information to get the context for each individual topic as you can see here and finally it gave me a report like this based on the information gathered from the get news tool here is the in-depth summary of the topics new AI program in Europe 2024 Trends report by handshake AI deep fakes and Global elections and it gives more information this is just the basic implementation of rag you can fine tune this further this is just to give you an idea I'm really excited about this I'm going to create more videos similar to this so stay tuned I hope you like this video do like share and subscribe and thanks for watching

Channel: Mervin Praison

Views: 8,766

Keywords: CrewAI RAG, AI News Agents, Automate, AI Agents, AI Assistants, Crew AI, Crew, AI, CrewAI, CrewAI Agents, CrewAI Assistants, Autonomous Agents, Autonomous Assistants, AGI, Crew AI Automate, CrewAI News Agents, CrewAI News Assistants, News, News API, AI News API, AI News, Breaking News, Automate News Agency, AI News Organsiation, AI News Company, News Agency, AI News Report, Automated News Report, RAG, retrieval augmented generation, CrewAI Retrieval Augmented Generation

Id: 77xSbC-9yn4

Length: 8min 11sec (491 seconds)

Published: Sat Feb 17 2024