Automatic1111 + Oobabooga = Exchange Images with your ChatBot!

hello and welcome to More nerdy rodent geekery do you know what is more fun than talking to a chatbot being able to exchange images with one obviously we're going to do this all for free on your local PC so let's get started first up you're going to need two things installed already up and running the automatic 1111 web interface for stable diffusion running in API mode is the very first thing you'll need and you'll also need this text generation web interface as well for the text generation now the next thing I did was to download so many models for text and image generation that my hard drive said no this is of course an optional step and you may not wish to devote your entire existence to this one task like I did some reasonable text generation models include Bloom that's quite a good one pythia 6.9 cerebrus GPT 2.7 is also very good those are all nice open models assuming I I can actually pronounce any of the names some of them which aren't so open which are also quite good llama and the Facebook opt if you've got a very small graphics card then the opt 1.3 B there from Facebook seems to work quite well I managed to get that one running along with stable diffusion so both of those at the same time and I was only using about nine gigs of vram now that you've got both of those running what you need is that special little link which ties them together the very bonding of text and image yes it's my really badly modified version of the SD API pictures extension which comes along with a stable diffusion prompt bot as well that I've lovingly crafted and now gift to all you lovely nerds for a grand total of free the link to my GitHub page with these two files is located in the very secret video description so pop down there and have a look for that all you need to do is copy those into your text generation web interface so that you have the SD API bot extension in the extensions directory and the stablebot character in your character's directory I will leave you to pick your own image for the bottom I trust you would choose very well as you can see this bot basically just provides some context to your chat sessions so you don't need any special fine-tuned prompt models or anything like that this will essentially work on any language model I wouldn't try using something like Galactica though you need something which can do a little bit of chatting maybe one day I'll go through that training tab we have at the top now do let me know down in the comments if anyone is interested in that but for now let's get to starting both of these engines first I tend to start the text generation web interface first and then stable diffusion just watch out as these both do use the same port ranges so starting at seven eight six oh you can enable the bot extension when you launch or via the interface mode that lists all of the available extensions right then let's get this bot going so down the bottom here I just expand all these various arrows you can see all the particular options that we've got the first one there is the SD API bot you want to activate the API integration so that it makes pictures and set it to whatever the address is there as mentioned mine is running on 7861 so just increasing that by one you've got some generation parameters there if you want to put in a prompt prefix if you want your own one in there absolutely fine stick it in there negative prompt again you can set your own negative prompt and image Dimensions there I'm just going to leave at the default 512. I've got a number of different Bots obviously I've been playing with this for quite a while now but the one that I'm giving to you for free is the best one here stablebot three now when I click on that it'll come up here and you should if you're really lucky get a picture that's because I've got a history of it so if I just clear that and confirm we'll start off there and we'll give it a fairly basic prompt let's get rid of that one and say show me a cat now it's it's going to think about it and it's not going to give you a picture back but to start off with it will give you some text so how do we turn that text into a prompt well go down here and if you press the force the next response to be a picture button you just have to click that once and then click regenerate instead you'll get a picture of a cat alrighty everything is working obviously the images generated will depend on which stable diffusion model you're using you can change that via the stable diffusion web interface only at the moment not in here right let's put this bot through its paces and I'll start off with some Noob level chat then move to intermediate level and finally some advanced level human AI interfacing right so down here at Noob level we're going to think like a stable diffusion prompt engineer which as everybody knows is a very prestigious job so let's start off with a really simple prompt we'll copy and paste this over here because I have prompts ready like that there could you generate a stable diffusion prompt for a photograph of a really evil kitten and the answer is yes it can there it is it's given me an evil kitten a black and white photograph print with sharp teeth and a malicious grin and all that sort of stuff so as you can see the prompt is quite descriptive about the sort of thing that it wants to generate again it's up to your stable diffusion model to interpret the words if you want to change the bot in any way at all it's very easy just go and change the text in there and you can add your own flares and styles and however you want to the prompt to come out so if you've got specific stable diffusion models no problem you'll just have to change the bot a little bit to use the specific words for your model all right let's say we can do something slightly different we'll have an anime art style cybernetic Barbarian there he is a cybernetic Barbarian apparently an anime art style he does look a little bit like that muscular Fierce and all that other stuff so it makes some really nice pictures all right so you could use this in all sorts of normal ways couldn't you like you do with stable diffusion so how about a design for a new men's t-shirt there we go got a men's t-shirt cool modern Sleek subtle and stylish that's quite a good t-shirt design I'd buy that myself I think or you could design a new piece of jewelry and see what this comes up with there a lovingly hand-crafted pendant that looks quite nice all right let's just do one more at Noob level something fairly standard what if I wanted a sunset in the style of a particular painter there you go we've got a sunset inspired by the particular Painter with all the various attributes assigned to their style I think that looks quite cool hopefully you get the idea ask for a thing and the image will be generated for you some extra descriptive words will be added onto your basic design and that often results in some fairly pleasing images now we enter the intermediate level and stop thinking like a prompt engineer we're using language models here which means we can do some quite interesting things such as treat it like a sort of Choose Your Own Adventure book I'm old so maybe you don't remember them but I do so I'm going to pretend I'm in one now where I'm in a room with three exits can you describe the room yes it can there it's a large room which is dimly lit with wooden floorboards interesting they seem quite what if I examine the floor what do you think let's let's put that in okay there we go copy paste if I examine the floor what do you think I would see oh it's a cracked floor uneven with age-old secrets and oh there's a hidden compartment and some loose boards you know what I'm going to do now don't you yes let's examine that hidden compartment we all want to know what's in there don't we oh okay there's a little box with all sorts of secret documents maps and treasures in there out of all of those I'm gonna examine that mysterious Trinket it's got strange markings and some sort of enigmatic design and a countdown to deadly consequences I think this object could be cursed and so it could go on obviously there won't be a storyline as such but you could carry on exploring whatever fictional world you care to imagine with this infinite Adventure okay so now it's time for mate my mind is just actually totally been blown level some of these smaller language models will likely give interesting results at this stage but if you've got a fairly General model with a few billion parameters or so it should do okay just to sort of carry on with the theme of games you can probe whatever language model you're using for the interesting things it may know for example you could do a sort of guessing game let's ask this AI to pick something at random and then describe it to us well we won't necessarily know for sure what the answer is in all cases it's pretty amazing that you can even do this at all let's try this prompt here how would you describe a random scene from the book Wind in the Willows as a stable diffusion prompt okay so there's the particular scene whether or not it actually relates to any passages in the book not too sure but maybe you know the book quite well and can guess it's a scene total this pandemonium wild animals eccentric characters and they are having a bit of a party all right let's try another one this time we'll do how would you summarize Shakespeare's Macbeth as a stable diffusion prompt yes we're taking the entire book summarizing it and turning it into a prompt and as you can see it's actually taken the word summary quite literally there and put it into the prompt but never mind but tragedy witchcraft prophecy power struggle all that sort of stuff as you can see it's it's done quite well but we've also got some other things up Our Sleeve as well we can always press the regenerate button again it's down to your particular stable diffusion model but if you don't like a particular picture just hit regenerate and you'll get a new one there we go I think that's done well we've sort of got a book and some curtains so you know it's a play and all that sort of thing yeah very nice prompt right let's get back to the gaming and this time I'm gonna pick a random scene from any 1960s movie and they're gonna try and describe it but without using the movie title In The Prompt do you think you'll be able to guess which movie this is so there's a Beachside bonfire some young lovers Stars twinkling overhead and waves crashing nearby Barefoot dancing and carefree laughter Timeless memories yeah I've no idea since it's a coming-of-age story so pick a coming-of-age story from the 1960s with that scene in it and you've probably got the right answer alright so I think you get the idea as to just exactly how imaginative you can be in some of these cases but as we're still in the advanced section let's kick it up a notch and enable the send pictures extension as well yes interface mode SD oh no the other one send pictures there we go apply and restart and then we'll be able to send this bot some pictures all right so when it restarts you'll just have to go back and select your stable bot again that's all right it's still enabled and it will remember where you've gone from but what we've got now is this send pictures option down the bottom let's do exactly that we'll drag a picture of someone with a beard there and see what happens and there we go it's taken that picture of a man with a beard turn the picture into text and then interpreted that text as a prompt so we've got a man with a beard standing beside another man with a beard for some reason and they're both having a great time at the barbershop so it's sort of taken the whole beard aspect and run with it quite a lot all right let's give it another picture the most obvious picture of course is one of a rodent let's see what happens there and there we go we've got a small white and gray Mouse not quite right but nevertheless it is the same sort of thing as the picture we got a rodent going on isn't that amazing and if you're looking for more information on how to get this text generation web interface up and running then look no further than this next video

Channel: Nerdy Rodent

Views: 34,165

Keywords: gpt4, chatgpt, llama, chatbot, image chatbot, chat with images, ai chat, ai chatbot, stable diffusion, chatgpt for stable diffusion, gpt-4, automatic1111, ai, artificial intelligence, machine learning, alpaca, gpt4all, Tutorial, Howto, Guide, How to, Custom chatbot, Text-generation-webui, Image chatbot, Image gpt, Exchange images with gpt, Oobabooga, Text-generation-webui images

Id: x5VUidEMDDI

Length: 13min 4sec (784 seconds)

Published: Thu Apr 06 2023