Send a Streaming Text Response in Microsoft Teams Chatbot using Microsoft Bot Framework SDK C#

hey everyone welcome back in this detailed video we going to experiment with a new feature functionality that we are going to implement for Microsoft teams bot using Microsoft bot framework SDK V4 C so as we all know with the enhancement of chat GPT open AI the response is quite fast if we see it as a streaming response but for Microsoft teams Channel until we generate the whole response we cannot show it to the user so this is where this experiment is coming into picture we will try to implement the streaming response for the Microsoft teams Channel and yes so first of all let me show you some of the quick recap that we'll be building buing in this detailed video let's say hi to the bot so as you see it has typed so let me ask it to generate just tell me a story that will be better it has a long response and also in the video you will find that we have implemented one word at a time thing but later after the video I tried experimenting it with a long text at a time yeah currently it is sending 30 words at a time but in the video you might see one word at a time but we have completed the complete implementation code you'll find the code in our repository website there's not much change there's only an counter I have also discuss how we can achieve that in the video and let me ask it to write some code okay so see here how quickly it is writing so with this implementation user need not wait for the complete response to be completed and then show it to the user you can show the response as quickly as possible okay this is a better user experience that that will be gamechanging in Microsoft teams bot I'm not sure if there is an existing functionality available I searched a lot but I could not find a redid Library well there is some library but I could not find the example of that how to utilize that so that is all let's get started hello everyone in this video we'll be exploring how we can send streaming data to Microsoft teams bot we'll be building this bot using Microsoft bot framework SDK C so with the enhancement of open AI chat GPT Services it is now crucial to implement the streaming responses so that we don't want the user to wait till the whole response is generated and then show it to the user with a streaming response we can show it as soon as some chunk of data is generated by chat jpt open AI uh as we all know that teams do not yet support the streaming responses but maybe in the future it might support but we have an alternative to implement that so let's try and get this done um first of all I'll be creating the Azure bot resource uh this process we have done numerous times times so let's quickly finish this up and on the visual studio side I have taken up the echobot template and also I converted this project to net a uh and also I will need to modify the program and startup CS file so so we will be getting rid of startup.cs file and keep all the dependency injections in the program.cs file currently the template do not support yet but most of the bot framework samples are structured that way already but that's fine and you must have received a notification that uh in May 2024 uh this Net 7 will will complete his standard life cycle basically it's ending so we have time till May 2024 to migrate all our applications to net a okay all right so coming back to this piece Azure open AI has release the client libraries that will be used for us to have better control over the streaming response data it will be difficult task for us to implement from the scratch uh using the API streaming point the custom API it will be better for us to use a client libraries so that we can just refer the respective properties and methods from the response to get our data okay we have our your bought resource ready we'll go to channels uh I'll not be implementing the Manifest uploading we'll directly go from here and start the communication let me run my angg rock one good feature that ngiro has released is you can now use uh you can now use custom domain name basically that domain name will be fixed this is much helpful you just uh start running your rro server and uh you did not come to your Azure resources to change the messaging endpoint that gets fixed currently as of now I need to replace this every time I open my ngok server okay okay perfect now let's go to uh before that let me just run this application and then we can modify it okay I'll go to channels and open in teams remember I'm not using uh I'm not using manifest to customize my chatbot I'm using directly opening the link so let's say hi okay uh I also forgot to include app ID and app Secrets let me quickly do that okay perfect let me restart okay there we have our response okay uh this is the echobot whatever I say it will reply me back with the same message here it is perfect now what what we'll do we'll go back to ecoboard let me increase the screen size so that it is visible to you all of you okay now now let's implement the client Library wow the page is really changed the uiux it's really looking nice just couple of minute back I when I opened it it was the old UI but now they changed it this is really looking nice amazing hopefully they changed it at all the places but how do I find the documentation link usually it will come at the bottom for all the Azure products now I need to check where I can find the documentation links so here it will be portal use case documentation okay here it is okay so we'll be calling the simple chat GPT without data but with VI data even they have released the client Library let me show you that quickly if I go to cop and here we have the client Library without streaming response and with streaming response but they haven't included the documentation for this extension piece this is an important piece that allows you to connect to your sources which is the Azure cognitive search as the source it's simple one but if you need to customize it that feat that documentation is still not available what does that mean is if you want to include the filters if you want to include the content Fields Vector fields that piece is still there in the client Library but documentation is not there how to use that so that's the reason I'll not go with the use your data I'll just go with a normal chat GPT that is chat completions C here we have without streaming response and here the we have with streaming response let me copy this piece of code and also let me install aure open AI remember it's still in preview uh so you'll not be able to see that preview package until you click on this include pre-release and select the package the Microsoft is constantly developing this within a just a month or two it came from BET beta point4 till point8 so they're continuously developing this perfect now we can import our name spaces yeah now aure open AI endpoint let's call our app settings. Json let me include this here similarly a z p i key okay let me get these two values from Azure open AI Studio I'll click on view code this is my endpoint this is my key perfect now I can just get this from configuration so let's import that okay perfect let's move this right a little bit so that it's properly arranged yeah okay um will not be implementing the state management or maintaining the history we already have created that video earlier with the use of storage account we can maintain the state or we can use as your Cosmos DB as Microsoft suggests that I'll be creating that video too how we can utilize Cosmos DB to maintain the state for now let's get rid of this and this this needs to be dynamic turn context do activity. text okay and let's make the max token as 800 I'll be making use of GPT 4 I think it is GPT 4 okay I have the turbo let me use the tobo all right and this is how you can fetch the streaming response yes okay message. content all right so how would you change that constantly the best way let's see how it behaves and then we can C okay yeah let's see how it behaves now what's the current behavior and then we can try to modify it okay bought encountered an error that's fine uh let's see what is the error uh let me get rid of this so that uh you might have to provide some condition here uh so so that you don't want to send the message error message to the end user I'm just copying this error code here and let's start let's say hi again okay resource not found resource not found where is second did I put all the details correctly GPT 35 turbo GPT 35 turbo let's see the deployments uh yes that's correct let's go to app settings and check okay got it got it this is the code that I had to use and I think documentation is still not updated okay till streaming chat completion I have to use this code let me go back uh let me get rid of this okay perfect now this will work because we haven't included any parameters okay yeah now let me put turn context. activity. text okay yes this should definitely be working now let's get the configuration from here put a slash okay that's fine open AI key we can get it from configuration perfect and perfect I guess this should work let's quickly check still bad request very strange well I tested this yesterday using my uh Blazer application this was working absolutely fine it's time for us to debug this thing uh before debugging let me change this activity Handler to teams activity Handler just in case it this showing me status as 200 okay the issue is with this one ah there's no issue with the Azure open AI there's issue with the response that we are trying to send okay got it let me come here and let's see okay it is null ah since it is okay got it since it is streaming response what we can do uh if string dot is null or empty message content if it is not null or empty then only show perfect hoping this should work okay there we have okay so the current behavior is hello how can I assist you today this is a streaming response but it is coming one at a time the all the messages so how do we try to work with this how do we try to work with this scenario so yesterday I while I was developing my blazer application I try to uh implement the streaming capability using the Blazer server application and yes the UI ux were not that great but the response was pretty good so let's try to implement that so this was my blazer code yesterday uh I just asked chat GPT to implement the HTML code for that uh because I am not expert in HTML CSS JavaScript so yeah it helped me with the HTML code so this is the way how we can utilize this so let me copy this piece of code and paste it here okay uh we do not need this one this one they specific to Blazer this one okay and we are done we are not done yet there still some work needs to be done okay uh basically we create a string and then we show that response to the user okay and that string keeps on appending with a new message content okay perfect and since we do not want to get into that issue the null so let's put a condition okay okay this seems to be fine no now how we will show this message okay we forgot about this one yeah we need to implement this messages okay uh we do not want that let me quick get rid of this and this time let me show this message okay a wait turn context yeah let's try this thing how it works then we can quickly modify it based on R this will still not work but I need to check how is the behavior Behavior yeah so this is a normal behavior until we have the whole message it will stay there okay now comes the magic that we all wanted to implement okay so what I'll do I will call something this I will call this statement update activity as sync this is there in teams bot and some of basically this functionality is valid for few of the channels one of them is Microsoft teams and slack other others I have not explored it yet so what this does is this replaces an existing activity yes before calling this set the ID of the replacement activity to the ID of the activity to replace not all channels support this operation for channels that don't this call May throw an exception but okay so so let's look at the documentation how to use this teams bot framework update activity okay to update an existing message pass a new activity object with the existing activity ID to the update activity a sync Method All right so uh of course we can use the update card as well but we'll first experiment with the text let's copy this piece of code but how do we get the activity ID how do we get the activity ID let's get rid of this statement from here let's copy this here paste it here and at the end Also let's paste it here okay turn context Dot activity dot reply to ID okay new activity ID reply to ID hoping this should work let's try that we're just experimenting as of now H bad request perfect uh the bad request can the reason being the bad request because this will be null reply to ID I'm assuming that this will be null but it should still send us the first message okay uh initially it will be null so we don't want to send that if bought length is greater than zero then only send this message okay let's start again okay it sent as hello the first response was hello as you can see here but later it did not send anything unhandled error activity ID cannot be null okay so now let's experiment is experiment it further to update an existing message pass a new activity object with existing activity ID to the update okay now that okay uh let's see the sample code that we will have better understanding okay update update there it is card activity async let's go to that card activity I think there it is update okay it is using cards send update [Music] card send update card okay activity do ID this we did that activity. ID return context reply to ID but it did not work because for text messages it is really difficult to get that ID H this also we can do messages updated reply activity okay let's do that but there's no new activity let's commment this out for now let's use this again let's try to use this H like ID cannot be null let's try this one okay forbidden at least some different error perfect perfect uh let's try now let's try to debug does it not allow me to update an activity that's strange okay we sent the first message the new text activity for the [Music] activity this is the activity ID update activity as saying why is forbidden okay let's ask to Google but framework update activity for bden okay somebody else has also got this error message my mistake was in a way of taking activity I which has to be updated D ID can be found from response only so here's a working code Old activity I sent the response ah water activity id response. id new activity new activity ID by the way you can do the same from dialogues using step cont Tex perfect perfect this guy is amazing let's try this okay old activity text it will be bought Builder do two string response it is old activity response bot this is the new act old activity ID the new activity the new activity since we sent that that since it is in Loop we wanted to send this updated activity ID okay let me do one thing let me add a flag this is for the first time first time is true is first time if it is first time we also need to specify that activity and also new activity this is if it is first time uh we need to send the old activity first time is false [Music] string hold activity ID okay so first time we send this now hold for the every other time we need to send this uh let's see if this logic is correct perfect perfect there we are no okay got it the reason being um well this need not be here well this will complete our whole message what we can do we can take this out well we are not using this old activity we can initialize it there and there itself yeah and let's initialize this old activity ID outside the first F so now when we come out we will use this okay when we come out we'll use this new activity final okay so that it will have the final message perfect let's say hi there we have let's ask something a large response uh let's say tell me a story perfect perfect well we need to think about the formatting aspect but this is pretty cool for uh one thing to take care this this speed is not that good until now we would have got the whole response quickly but the speed is still not that good but it's much better than waiting because the user can keep on reading the response until chatbot has finished okay let's ask something uh coding question we can always optimize the response currently what it is doing it is going step by step like one word at a time what we can do we can skip some of the words and then type it fully I think this formatting is taken care of okay that was good and let's try to see how we can optimize it based on the number of characters that will be updated so here let's say uh let's put it at the five times uh five characters at a time that will be better I think for this video this is fine I have an idea about utilizing how we can speed up this response so let me show you the idea and then uh if in future we can Implement that so what I was about to do I was creating a list of string and then uh I will be adding all the responses in that list tring till five responses and I will be checking if the response is exactly five then send the message else keep on piling up the response so when the response is sent clear the list item this way the response that will be getting will be five words at a time that will be much faster I think the one word is also nice okay thanks everyone

Channel: Dewiride Technologies

Views: 900

Keywords: ChatGPT, bot framework, C#, OpenAI, GPT-4, conversational AI, tutorial, chatbot development, Microsoft Bot Framework SDK, chatgpt, openai chat completion api, ai chatbot, chatbot tutorial, beginner tutorial, microsoft bot framework, openai, bot development, chatbot, bot, open ai, ai, ai bots, chatgpt tutorial, azure bot service, chatgpt api, build chatbot, create chatbot, create ai bots, microsoft teams bot, teams bot, azure openai, streaming bot, streaming teams bot, ms teams

Id: JR7SA5Zh8mA

Length: 48min 2sec (2882 seconds)

Published: Fri Sep 29 2023