How to Create 3D Virtual Assistants or Digital Humans - Ultimate Chatbot Tutorial

[Music] [Music] [Applause] so [Applause] [Music] [Applause] so [Music] [Applause] [Music] hey everyone welcome back to another episode of tech with search my name is sachin a google developer expert and in today's video we'll learn how to extend our chat bots to virtual assistants or digital humans so without further ado let's get started in our previous tutorials we learned how to build a chatbot from scratch we also learned how to use it for understand more about intense entities and fulfillments we connected faq documents through knowledge bases and connectors we also learned how to integrate our chatbot to platforms like google assistant your own website teleflow messenger and the chatbot that we've built so far the functionality and the features of that is basically it it's it is an appointment scheduler which will basically get the information from the user and then integrate it direct directly to your google calendar and apart from that we went a step further and integrated a chatbot to the entire array of platforms that dial up flow supports and does not support as well so things like telephony we were able to directly integrate our chatbot to through the telephone you can pick up the phone line and call a number and integrate interact with your chatbot as usual we also looked at how you can integrate it to facebook your facebook page through facebook messenger whatsapp slack telegram line so we've already integrated this with a lot of these platforms and finally in our last episode we learned how we can spice up our chatbots and the whole immersive experience provide immersive experience for the user by using rich responses and ssml so for today's episode we have a very special episode and this is going to be one of the last sessions of 2020 so what i wanted to do today is explore some of the key concepts of digital humans so in today's episode we'll learn how we can uh what are the different platforms that's available today to explore uh digital humans how we can extend our existing chatbot that we built so far as part of the ultimate chatbot tutorial series to virtual assistants and how you can create your own 3d virtual assistants or digital humans and also add behaviors so depending on how your chatbot responds uh chatbot responses you can also add some kind of emotions and behaviors to your virtual assistant and finally we'll have a complete review of what we've accomplished as part of the ultimate chat or tutorial series so if you're new to the channel make sure to subscribe as well as hit the bell icon so that you can be notified for future videos we do have these weekly sessions happening every week so i'd love to have more of you interacting live as well as do let let us know if there are specific softwares or applications that you have covered uh integrating with dialogflow leave them in the comments section and we'll be more than happy to review it as a community so without further ado let's get started so just to kind of go back to where we started so first thing that you need to go to is console.dialogflow.com so here for people who are joining us for the very first time we've already created our chatbot as part of the previous episodes you can still continue to watch this and see what are the different techniques that we can use to integrate this with 3d virtual avatars or virtual assistants [Music] so step one is go to console.dialogflow.com it will take you to the dialogflow console and within the dialogflow console you'll have what you can see is i've already created a project and an agent so the agent name is called chatbot my experience and within this what we're doing is whenever you interact with the chatbot so for example when we say hi it's gonna respond back with a default response which says hi i can help you set up an appointment and help you learn more about the coronavirus and then from there you can say set up an appointment but let's just say set an appointment tomorrow it's gonna ask me step by step questions i say 8 pm what kind of services are you looking for dental service [Music] and we can see that okay let me see if we can fit you in december 31st 8 pm is fine so it has successfully understood understood that there's a time slot available gathered all the information from my query and then we have already set that as part of the calendar so if i go into the calendar and check the time slot for tomorrow so you can see between eight to nine there's already a dental service appointment scheduled perfect now while exploring these digital humans and especially this is something that i came across while i was watching one of the youtube series uh regarding artificial intelligence and there was a company which actually uh was highlighted which is called soul machines so this is a company who is actually working on building digital humans so the idea is you can interact with these digital avatars or digital people and have the same kind of voice experience that you have a conversational experience that you have with the chatbot but here it's extending just not with just chat and voice you also get emotions and life feedback from reaction just like how you interact with a human right so that's why they call it digital people or digital humans now there's a lot of use cases that they have highlighted within this website but what i would like to see is show you a demo of what they have accomplished and again just a disclaimer this is not a sponsored video as part of this episode i just wanted to see what are the third-party applications out there who are doing certain applications and based on different frameworks now once we see what soul machine does we will also see another platform where we can directly integrate our chatbot that we have built today into one of these digital humans so let's actually start with the soul machines so you you do have an option to try now and let's talk to sam as an example and this is actually a pretty cool demo of how they blend in multiple things hi i'm sam your guide to soul machines what's your name [Music] sachin great nice to meet you sutton have you spoken to a digital person like me before no [Music] nope okay would you like to play a fun little game and learn how to interact with me sure [Music] sure [Music] yes great the game is played simply by responding to me easy as that ready to play yes all right so first here are some tips that'll help me respond better to you make sure the room is quiet you're close to your mic and sitting in front of your camera so sudden if i were to ask you what two plus two is you would say it's full i can't decide for you so sudden if i were to ask you what two plus two is you would say it will be number four you got it but i'm doing more than listening to you i'm also looking at you i'm able to read your face using your camera let's try it out go ahead and flash a big smile and say cheese cheese i wasn't able to find your smile but that's okay if you're noticing a delay before my answers it's because i'm thinking i need to do that sometimes and hints like these can appear on screen to prompt you for an answer how do you think the game's going so far it's wonderful i'm happy to hear it see that text icon at the bottom of your screen you use it to type your answers if you don't feel like speaking or if i mishear you let's try it out could you type your favorite color in a rainbow violet awesome and one final note i'm not google i can only answer questions about myself and soul machines and that's it like you said sutton easy as pie thanks for playing this game with me great so as you can see it did get my name absolutely wrong so instead of sachin it was certain but you can see the kind of experience that you have and one of the cool things that soul machine does really well and as an example that we've kind of touched upon in our previous tutorials is the conversational design as well you can see how we've kind of interacted with the chatbot and how the chatbot replies back in a way that it's asking you for a specific question or leading you to a specific action so in that way what happens is the conversation keeps rolling and it's not an open-ended question so these are some of the good design guidelines when it comes to conversational design so i hope you guys enjoyed that one and you can actually go and try out based on different industries you have a lot more so for example when you're talking about healthcare there's one that they have done in terms of as part of the who so you can check those out when you when you guys get time so do try them out so speaking of soul machines and as a company and trying this demo i was actually exploring are there any other companies which actually does some integration with dialogflow to have some kind of virtual assistants or digital avatars so if you guys know of any other companies who are doing this or if you know if you build any applications or platforms similar to this do let me know in the comment section and i'll be more than happy to review and come and review and share that same thing with the community so the next thing that we'll do is try and build a similar experience through a different platform called unique so step one is go to unique dot io so here what is interesting about this specific platform is it has direct integration across multiple uh platforms so in this dialogflow is one of them so it is easy for you guys to actually integrate your existing chatbots that you have built in dialog flow into uh through this platform so you have supercharged efficiency digital workforce you can see how these can be used in different scenarios in different use cases like the whole customer experience so you can see the different kind of digital humans based on different use cases so one as a customer assistant for health insurance a sales concert um financial advisor healthcare coach digital influencer and so on so again if you have already built something similar to this do leave them in the comment section or if you've come across some of uh something similar to this as well let let us know in the comment section and if you're interested in how you can build something like this from scratch also let me know so that we can actually see how we can build this together okay so the first thing that you need to go is go to unique.io or digitalhumans.com you get a trial version and again one thing to keep note is this you get a 14 day trial edition or trial version this is not a free platform of you'll have to pay for this and if you are a company or a startup and if you have the bandwidth for using a service like this feel free to use it but again there is a pricing that comes along with this okay so once you are signed up for the trial account once you've logged in so you can see that i've logged in over here with the trial account and then once that is done you'll find the specific dashboard so with the dashboard you can see that you can share your persona that you have created you can connect your connect to your conversational platform you can also learn more about how you can add emotions and behaviors through their docs and finally you can also deploy your digital humans across multiple platforms or mediums okay so to get started the first thing that we'll do is you can go to personas and create a new persona here so you have different persona depending on what you need you can choose one and hit next you can also choose a background of your choice okay let's choose something like this hit next and give your person a name so you can say john or anything so hit next choose a language and choose a voice hi there i'm a digital human it's great to meet you hi there i'm a digital human it's great to meet you so depending on your persona that you want your customers to experience you can decide how your persona should look like what the voice should look like how what tone of the voice should look like a sound like and so on so hi there i'm a digital human it's great to meet you in this case i'm just gonna go with the second one hit start so in this um hello there it's so good to meet you thanks for spinning me up you have no idea how cramped it is in the cloud you probably have lots of questions for me i'm here to help you with some of those through a guided conversation or we could just chat you can select an experience when you're ready so let me go and end the conversation so you can see how easy it was to just choose your persona and configure that right so the next thing that we need to do is we've already created our chatbot which basically schedules an appointment for us through dialogflow how can we link this with our conversational platform uh with the unique platform or the digitalhumans.com so the way you how the way you do that is first you can go to the documentation and look for integration with dialogflow es so under conversational platform integrations click on google dialogflow es and here they provide you the step-by-step instructions on how you can integrate it so i'm just going to show you the same thing over here so the first thing that you need to do is go back to the dialogflow console under your agent click on the gear icon and once you go to the settings what you can do is we need the project id which is going to be this so copy this information and you can come back to your persona so for example if i click on personas i've already integrated one so if i click on edit you will see that i've pasted the project id over here okay so once your persona has created you can paste your project id here again for you to get the project id you can get it from your dialog flow agent click on the gear icon and under project id copy this and paste it over here and the next thing for us to for this platform to successfully connect with your dialogflow platform it needs a service account access and for you to generate that service account all you need to do is come back to the dialog flow console and click on this google cloud link if you have multiple accounts like how i have here you can choose make sure to choose the right one that you're using that you have used with the dialogflow once you have done that what's interesting over here is you can go to the navigation and under identity and security go and create a service account so click on service account and you can create a service account so here you can see for multiple integrations i've created one so for service account you can just go ahead and give it a service let's say dialog flow digital unit create it and you can also grant the access to the thing so the way how you do that is go to project you can give based on different options you can give different roles so if you want the service account that you have created to have only very specific access you can choose that based on this role over here and hit done [Music] once that is done you can come here you can see this is the one that we just created and even before we do this one of the most important things that you need to do is if you have not enabled the api dialog flow api yet you need to do that so the way how you do that is go back to the navigation menu and search for apis and services go to library and search for dialogflow api you need to make sure this dialogflow api is enabled if you have not done this this whole integration won't work so we've already covered some of these in our previous tutorials and for people who are joining us for the very first time directly in this episode uh make sure that you have your dialogflow api for the project that you have done in your google cloud enable this so i've already enabled it so we are all good in those terms so let's go back to under identity and security go to service accounts and if you look at what we need to do next is we've already created a service account but we need a private key we need to generate a key so go ahead and create a key choose json [Music] and this will automatically download that specific json or the private key to your computer so keep this safe what you need to then do is copy this json or move this json and add it directly over here so basically you need to upload the file over here and move it to the key that we've just generated so in my case i've already uploaded that so if you go back to the documentation you can see the three major things that they need is the project id language and the service account private key so using this service account private key is how they communicate and integrate directly with dialogflow okay okay okay so perfect so let's then just give this a shot we've already done this we've already chosen the language and the settings and voice save it so go back to personas or go back to dashboard you can also share your post persona with other people so i've given the name that i've set up as eva so copy the link let's open up in a new tab [Music] [Music] so if you have any questions or queries do leave them in the comment section below hi i can help you set up an appointment and help you learn more about the coronavirus so as you can see we have successfully integrated a chatbot to a digital human right so here is a much more immersive experience you feel like you're actually talking to a human or at least close to what we kind of have as a natural experience now with that being said let's actually go ahead and try this out the entire experience so instead of in our previous tutorials uh apart from the google assistant where we interacted with voice all the other platforms were basically via chat so you had to type in the message to interact with so here you also have the option to type as well as chat i would like to set up an appointment at what time 6 pm when do you need this [Music] tomorrow what kind of services are you looking for i would like to have the dental service [Music] okay let me see if we can fit you in december 31st 6 pm is fine so you can see the emotions and other things as well now we've not added any specific emotions to this as part of the platform specific emotions or behaviors that the digitalhumans.com provides but let's also check what happens when we come back to our dialog flow right now so let's go back to intents and under welcome intents this is the response that we initially got from from dialogflow right and let's also go back to our calendar and check whether an appointment has been set yes between six to seven an appointment has already been set perfect so let's actually take this a step further and add some emotions or behaviors to our digital human so the way how you do that is again as part of the documentation you can see they have something called as emotion tags or you know unique tags so you can see this this unique happy is just like how we learned ssml you can also add these tags and have the text in between them so that it will automatically render based on their platform so the way how the whole thing works is whenever you have this response since we have the service account integrated with the unique platform they get all of these responses via json so through the dialogflow api and once they receive it in their platform they would render this specific text that they get so in this case like my name is sophie what's yours and this is wrapped around the specific tag and based on this tag they would make sure that a specific emotion or behavior is shown on the additional human right so that's how they kind of achieve this from a high level standpoint so let's go ahead and try and add some of these tags and see how we can add different behaviors so these are called the behavior tags so right now they have happy empathetic excited joking agreeing or enquiring so let's try that out so the first thing is let's start with the happy tag let's go back to our dialog flow let's keep the same thing i'm gonna remove this just gonna say click on save so anything any text wrapped around around the specific behavior tag it will automatically reflect in your digital human so let's go ahead and save this and also give this a try over here so let me refresh and for people who are joining us i can help you set up an appointment and help you learn more about the coronavirus make sure to subscribe as well as hit the bell icon so that you can be notified for future videos so as you can see since we added that behavior tag in the beginning you can see the emotions or the behavior a happy face an expression that gets related back to your digital human let's also try out different things so let's say as per the documentation you have empathetic as one of it excited as also one of it so let's say enquiring so let's go back and since in our appointment scheduler we already asked these questions let's go and see if it can handle these problems directly from here so these are all based on enquiry so the chatbot is or the bot is actually asking us these enquiry questions like what time when do you need this what is the dental sir what service that you need so all of that is being handled using these behavior tags so we've wrapped around our text with these behavior tags and then let's go ahead and save this [Music] and let's refresh this and try out how our chatbot behaves right now [Music] hi i can help you set up an appointment and help you learn more about the coronavirus i would like to set up an appointment at what time 10 pm when do you need this on friday what kind of services are you looking for i'm looking for the dental service okay let me see if we can fit you in january 1st 10 pm is fine perfect so as you can see i'm not sure whether you guys have noticed it but you can see how the subtle changes in the expressions or the behavior based on wherever we've added our tags so again uh unique or the digitalhuman.com these guys have done a wonderful job at how easy it is to kind of integrate with dialog flow and what you can explore is also some of the other changes like so for example um joking so let's copy this and just see from as a demo a standpoint let's just copy this as a response and see how that behaves okay and let's try this again so here we've used the joking tag behavior tag and then we have a text wrapped around it let's see how the character behaves or changes based on that what do you call two monkeys that share an amazon account primates okay so you can see how the expression changed based on you know that joking tag so these are wonderful examples of how you can do and for people who are actually wondering how you can actually build this by yourself let me know if this is something interesting that you'd like to explore but behind the scenes what you need to look at is how you can build an avatar and based on the avatar if you can pass text messages and can come using text to speech to text if you can just pass text and if you can simulate the whole experience over here then the rest of the things are easy because you can use the dialog flow api and based on the request and response that the dialogflow api passes to your platform you can extract the information and especially things like these special tags or anything that you define as part of your platform you can extract that and can add these behavior changes this this can be done through multiple ways so this is such an example so i have some questions coming through from neeraj please tell us where book right now if it was possible yes you can everything that's happening right now is also going through the web book so we have our own web book that's integrated with our calendar that's how the calendar appointments are being set so for example the last conversation that i had was on friday 10 p.m so that's already being done right so you can do that and one of the things that i've also seen is as part of their documentation is you can also add html elements so under custom payload like how we've added in our previous tutorials you can add html string and that would basically appear here as visual queues so do check them out and see how you can build these again you have a 14-day trial and as i mentioned earlier this is not a free platform if you actually go to unique dot io and search for the pricing depending on your use case you can choose but again this is not for everyone as i mentioned so you can always go and build your own if you're interested but if you think that this is a platform that you can already start integrating and get all of this done quickly you have this advantage of using this so that was a quick uh short review of uh how we can add digital humans or virtual assistants and finally what i would like to show is for people who are interested in google assistant you should actually check out interactive canvas so using interactive canvas is a wonderful way so it's a framework built on google assistant you can use simple html css and javascript to render your visual content so you can use both voice as well as use the display as visual cues okay so in this case what you can do is come down and you have different tutorials based on this you can also refer to my website which is techwithsuch.com i have i have an example of interactive canvas uh step-by-step step tutorials we'll also be covering interactive canvas as part of our upcoming sessions as well so uh this is gonna be our last session for 2020. we have a lot more coming uh for the next year be it cloud machine learning we're going to combine and blend all of these things together and to show you an example of interactive canvas i'd like to show you a video of something similar to what we've done right now hey google talk to my test app [Music] getting the test version of my test app so this is using interactive canvas [Music] so you can add sound effects through ssl okay you're back let's get the game rolling here is your first question [Music] which is the third planet from the sun planet earth that is correct it is earth planet earth as the third planet from the sun which planet in our solar system takes about 11.9 years to orbit the sun is it jupiter that is correct it is jupiter so that was an example of how you can build uh an interactive experience using a robot and that robot was rendered using 3js javascript library so you guys can actually go and give a similar experience to that it's not necessary that you need to have some kind of a digital human as such but you can also kind of use it use existing things like html javascript and css as developers there's something very basic that most of us are familiar with so we can always use our existing knowledge with html css and javascript build the whole experience and use uh the interactive canvas api to build such immersive and visual experiences so with that i hope you guys all enjoyed this episode and if you are new to the channel once again and if you like this video make sure to like the video as well as subscribe to the channel and see you all in the next one take care and stay safe [Music] you

Channel: TECH WITH SACH

Views: 7,527

Rating: 4.9800997 out of 5

Keywords: How to Create 3D Virtual Assistants or Digital Humans, create digital humans, create your own virutal assistant, create ai chatbot using dialogflow, how to create a 3d ai chatbot using dialogflow, dialogflow chatbot tutorial, dialogflow tutorials: how to create 3d virtual assistants using dialogflow, create live chatbot, dialogflow tutorial, digital humans, uneeq digital humans, soul machines, 3d avatar chatbot, google assistant, interactive canvas, dialogflow integration, bot, ml

Id: gjCqS91Vl98

Length: 43min 30sec (2610 seconds)

Published: Wed Dec 30 2020