15.9: Amazon EC2 Deployment - Twitter Bot Tutorial

hello in this video I'm going to look at how you can deploy a Twitter bot written in nodejs to an Amazon ec2 instance in particular one of the reasons I want to use an Amazon ec2 instance in this case is because not only is my Twitter bot written in nodejs but my Twitter bot triggers a processing sketch which is a Java program to create an image and that image is posted to Twitter so if you want to make a image producing Twitter bot this is one solution for you this is not the be-all end-all solution of course there's lots of ways to write Twitter bots and there's lots of ways to make images but you can see here so I'm in a moment I'm going to start going through those steps but let's just kind of get a lay of the land here for a second so here you can see this is a particular trigger bot it posts an image made from processing you can see the image here this is the processing sketch if I run the processing sketch here you can see it just draws this particular image it also saves it to a file and exits because here now is the code you can see the code in node is triggering that processing sketch here and now if I run that code the image bot is starting it says it worked and if I go back to Twitter we can see here I got another one now these two images are different even though they look remarkably the same remarkably similar but they are they are different right someone's doing a quick pixel analysis they're different they have to be different otherwise my whole understanding of the way the universe works is faulty ok so now we see this is how you run this stuff locally this is what I did in all the previous videos in this playlist about node Twitter bot things now we need to look at how do we how do we deploy this stuff to an amazon ec2 instance you know basically you know any of any Linux Linux server you could follow these same steps and I'm just going to happen to use Amazon and I was able to do this with a free account now at some point you might discover you have to pay but for the most part seems like you can run this using free credits at least for quite a long time okay so I do happen to have these notes online on a web page which will be linked to in the description of this video which presumably you are watching it some other time than the actual moment that I'm speaking right now but the first thing I'm going to do here is go to this link Amazon ec2 but actually I can just type in Amazon ec2 probably into Google and I'm going to even just go down here and click on the first result and what I want to do is sign into the console the console is where I'm going to get to configure my server so that's the first thing I want to do sign into the console and then I have to enter in my password which is with the full robot and I told one chrome to save it okay ah I've presented with this terrifying page of like 50 zillion things we could do I there's so much here I have no idea what any of it does but up here in the top left ec2 virtual servers in the cloud floating in the air so those servers which are more likely in an underground bunker somewhere not in a cloud but this is what I want I want to run a server somewhere else where I can run this particular bottle so I'm going to click on ec2 and I get an ec2 management console and oh good oh no I do so but I have I currently have one running instance which is I guess probably some previous test that I did at another time but I'm going to just click Launch instance and now I'm presented also with a terrifying set of objects of of objects of options so I'm going to go back to my notes page where I said this is the one that I happen to use so there's different versions of Linux operating systems and different kinds of and the key things that I'm looking for one is free to your eligible and see if I can get this running on the free tier the other is I need one that can also not just run node run Java it so happens that this Mbutu server 14.04 it happens to also be able to run java I also want to make sure I pick general purpose free tier micro and then set review and launch so I'm going to do that I'm going to look for that one which is this one I'm going to hit select over here I don't know if you guys can see this I should make that but boon to Server 1404 I'm gonna hit select a general purpose T to free TR eligible I'm going to hit review and launch and now it is launched great oh no lunch now I'm going to hit lunch and ah okay so ah so here's so if okay so here's the deal this is the weirdest part of working with Amazon ec2 is how you're going to now authenticate and login to it and you need to create this secret file that holds the keys to your server now apparently I've been here before and I've made a bunch of these because I was showing different examples so you likely won't have any previous keys you can use so but I'm going to create a new key pair so I need to give it a name and I'm going to call it coding rainbow if you can see that and I'm gonna then click download key pair Oh yikes ah what happened I'm just who make like a Chris I've completely lost my ability to zoom effectively so I'm gonna hit download key pair and we should see hopefully that ah you can see here I'm standing in front of it but down here under my downloads in the downloads there is now this file coding rainbow dot p.m. you must have this file I now need that file always and forever to log into this ec2 instance in a variety of different ways that I'll show you okay so I have that file now and now I should be able to now finally say launch instances now that I have that file I can say launch instances and waiting your instances are now launching and now I can go to view instances and I don't not worrying about that right now so you can see this is clearly one that I made a while ago when I was doing another tutorial or another test or and you can see this one hasn't actually completely launched so this is the problem Vicky's videos in real time where I sort of uh I just stayed here like completely awkwardly like dumbfounded I don't know what to do it's pending let's go to the next steps and when we come back hopefully it'll be hopeful it'll be working so okay so I followed these steps I did this part now aha okay so what I need to do now is use that key that file to be able to log into that instance so the way that the first thing I need to do is actually change the files permissions so files on computers have permissions some files are like locked you can't edit them some files are applications that can be executable this particular key file needs a CH stands for like change modify what is it for somebody doesn't think somebody right I'll do an annotation here is that but I I need to run this command to change its file permissions so what I'm going to do is the first thing I need to do is I need to go to where that file is so actually I'm just going to put that file on the desktop you can come up with a better place to put it and I'm going to go to the desktop and I can see there is that file and I guess I'm zoomed in or something so now what I want to do is oops type in this particular command chmod 400 and then that filename which is was coding rainbow PE M so that's the first thing I've done I've now changed the file permissions to 400 for that particular file and then now I can you do something called SSH SSH secure shell so basically what I want to do is in the same way that I have my I'm here in terminal on my computer and I can look at the file by saying LS I could run the files by saying like node what I want to do is be able to remote into that Amazon ec2 instance so that the terminal I'm looking at the commands I'm are actually not running on my laptop which is here yeah it's dangerous to pick it up not running on this laptop but actually the commands are executing on the server floating in the cloud that's buried in the underground bunker okay so let's see here if ah so me rambling for a long period of time we're at nine minutes here has actually caused this this instance to be running I don't know why it doesn't have a name here let me give it a name so and one thing I can do here if i zoom in and right-click see this connect option this is going to be really useful to me because actually it gives me all of right this is the command it's got I've got the command right here this is like the public this is like the IP address essentially of that instance and then here it's actually giving me the exact command to log in to it so I don't even need to figure out what it is I can just copy this and go back to terminal and paste that in here hit enter and okay this is the first time that I'm connecting so my computer sings I don't know about this I haven't connected this place before is anything something that I should be afraid of you it's nothing so I should hit type yes and now it's added and then I just stood here and I hope and wait and I hope that the internet will oblige me and connect ah there we go so look at that you can now see here that I am connected I have logged in under my new name Vudu to my new address IP 172 31 34 179 so now I can do things like say PWD with print working directory here I am in home slash boom - I can type LS and I can see there's nothing there so now this is great right this bot which I have here locally on my computer I want to be able to get it over here on my Amazon ec2 instance so how do I upload files there so the way one way to upload files to your Amazon ec2 instance is with a FTP or File Transfer Protocol secure FTP so the way that I could do that is you could use any any FTP software and all sorts of define a way to do it through the web it's through a browser but I'm going to use something called cyberduck which you can google and download I already have it installed here and so I'm going to run cyberduck apparently hi and what I'm going to do here I have a cyberduck open what I want to hit is this open connection so I click open connection and now I have this window where what I need to do is put in all the information for that Amazon ec2 instance so first thing I want to do is make sure I do SFTP it's got to be secure shell File Transfer Protocol and then the server address if I go back to here should be this right this is connect to your instance using the Public DNS it's this so I'm going to copy this and go back to cyberduck paste that in here now there's this interesting question here which is what's my username and password well I don't have a username and password I have this weird coding rainbow dot p.m. private key file it turns out in cyberduck under here under more options I can click that and I can say use public key authentication so I click that and now I go to the desktop and I select this file and now I now have this file selected as my way of authenticating and I should be able to just say connect oh maybe I need to write new boon to here just in the username and it works ok whew so that was one little step that I had personally forgotten that I need to put in the username I need to say boon to but now I just have this window here right this is now a view into home slash and boo - so what do I want to do I want to go and I want to just go here this is this folder here node 5 is where I have all of my stuff and I'm actually just I'm just going to take bot config and package.json I'm just going to upload those files right now so I can drag them in here drag those files into here and they're now uploading a little windows telling me progress they're transferring I can see that those files are listed here right now and now I can go back here and I can say LS and you can see those files are there yay these things are working so now running out of breath here now I can type in node and you know this is going to not work because I missed a set of steps here but node bot J s and so what's happening here the program node can be found in the following packages node nodejs legacy so and then saying sudo apt-get install selected package so one thing we first have to do is at least configure node to work the way that we want it to work and I believe that I have mentioned that here and these are some steps that you can do so let's actually just run through these steps the first thing I want to do is call what do sudo apt-get update so what that does is just says this apt-get thing which is a tool to like get things like node another software I want to make sure it's updated and has all of the most recent stuff so that's done then the next thing I want to do is install node J s which say sudo apt-get install no - Jas so I'm going to run that my audio is still working up this is asking me if I want to use the disk space which of course I do so I'm going to run and install nodejs which is great then I also want to make sure I have NPM installed which is node package manager which you might remember from the previous node tutorials and this is a 130 megabytes of disk space so hopefully this doesn't take too long so this is installing now the next step while it's installing let's talk about the next step for a second this is now a command to make sure that the particular version of node that I installed is now the actual version of node that will run when I type in the node command so I am going to grab this command here and hope that this is done it's done and I'm going to because watch what happens if I just type node right now it doesn't actually know what note it can be found in it's like well which one should I run no js' legacy or this one node so I want to make sure that I run this particular one with that command and now when I type node you'll see I'm in node so now I have no running on my Amazon ec2 instance and I can oh if I say clear here oh it does still work I can say node bot J s now first error I get is it cannot find module twit this is going to be notice when I uploaded my files remember how I said I upload the files I didn't upload the directory that said node modules I could have uploaded that directory but it makes more sense for me to just let those modules get installed by Amazon itself so I can just type in npm install and it's now going to install all the modules that are needed for this particular bot we went through this locally now I'm doing this on the server so if if we looked here and I said I refreshed you can see that now there is a directory called node modules which has for example that twit module in it so now I should be able say node bot J s and it will run now here's the thing it runs so these are all the steps you need if you just have a plain old bot that only runs a node but the reason why I got an error now is it can't find that image file output dot PNG the reason why I can't find that image file output dot PNG is because that the image file that's created by the processing sketch so while this part of the video I've now concluded like you could if you just have a node dot J a spot you're kind of done although there's another step of getting it to run after you out which I will get to let's but let's get the let's get the image itself to let's get the image making processing sketch to run as well so let's look at how we do that and there's a lot of like sort of awfully scary looking weird commands to do that okay oh but actually so yeah I'll come back to the forever thing okay so the first thing we need to do is make sure Java is on our server so first thing I want to do is install java which i want to say apt-get install default JRE so I'm going to put that in and run that and it's going to take 209 megabytes of disk spacing to say yes so now I'm installing Java I'm going to wait for that to install let's go back while we're doing that - the next step the next step okay so look at all these like terrifyingly weird looking commands so what's the weird thing that has to happen when you work when you work with processing on a computer your computer is typically connected to a monitor a screen and you're seeing the output of processing on that screen your Amazon ec2 instance has no screen it has no GUI interface it's just a server that you can SSH into or FTP files to but it's known as headless meaning processing actually when it runs it breaks because it's like I don't know what graphics thing to draw anything - there is no graphics thing so these next steps are steps that create a fake display essentially the ec2 instance has this display you're tricking it into thinking that it is connected to a display because we don't need to see it we just want the graphics to be rented to a file so the first thing I'm going to do is install something called X VfB which you could Google to find out what it is I'm going to run this particular command which will install this tool for a fake display the other thing I'm going to do is give it a now I'm going to give it an actual screen which is has a resolution of 1024 by 768 why not so I will do that I didn't give some weird little errors but you know big deal so this seems to be running a process that I need to like now quit out of so I'm going to ctrl C to do that and then I'm okay so this was this was the command that creates that screen and that screen was there only while it's running but I also now need to export it and assign it to be display one so actually these commands are no good so this is the command let me let me explain that again so I'm going to run this command right and that screen now exists for processing to draw into but I can't type in any other commands here because it's like running so what I needed to do is I needed to run in the background and a command for having things run in the background is no hub so I'm actually going to say this now and now you can see I'm back to my terminal but that screen is now created and I also just want to assign it display number one that's what processing will draw to display number one so that's done so now we're in good shape now here's the thing if we go back to the code you'll notice here this is the code that I ran locally on my laptop for triggering processing to run processing - Java but actually now that I'm on a Amazon server boy this is on 20 minutes it is anybody still watching as 20 minutes into a tutorial uploading stuff to Amazon ec2 but I have now I'm now on an Amazon ec2 instance which is a Linux machine so there's actually a different way that I could do this I can go to my processing sketch and I can go under file export application so export application is something you can always do and it opens up a little window and what I want to actually do is check Linux ok I want to export it as a Linux application now there's this embed Java option which I'm kind of confused whether I want to select it I I don't recall whether I need to select that or not let's try unselecting it I probably should keep it selected but let's try unselecting it and I'm going to hit export so you can see here now I have the Linux application incidentally here's a version that you could run on a Raspberry Pi which we'll get to later so now I should be able to go back and upload this to okay so I'm going to have this application which I want to include so oh now boy this tutorial is not going very well that light went off camera went off I don't know why this light keeps going off so I'm going to now now what I need to do is I need to take this folder and I'm going to upload that folder to books my server so I'm going to upload that here so I'm uploading that Linux application dumped unta it's there now I'm also going to rename this file this folder and call it rainbow and I want to I do want to do that so you can see here the application is now here in this particular folder so the first thing we need to do is figure out does this even work so remember here I am now on my I'm now logged in to the server and you can see there is a directory called rainbow so I can go into that directory we can see what's there this is actually the application and the way in Linux the way that you execute something is with dot slash rainbow and it's going to say permission denied so I did miss a step there which is that the permissions of the file I have to set the permissions to be executable which I can do by saying chmod 777 rainbow and now I should be able to say dot slash rainbow and run it and it did something it finished and now let's type LS and we can see look at that there is a file in there now called output dot PNG so if I go in here and hit refresh and we go grab this file and I download it to my desktop and I take a look at it you can see this is the image that was now generated on the server so now we have the processing sketch as a exported Linux application running on the Amazon ec2 instance ah well this is I know this is like 20-some minutes into this long tutorial we're so close now to actually making the thing run it's very exciting ok so now if I go back to here one thing you'll notice I'm going to go back to the code is remember this this was the command that I had for running on my Mac to execute a processing sketch using processing command line processing - Java but now that it's a Linux application the only thing I need to do is actually just I change that command to this dot slash rainbow only that's not exactly right because it's in a different directory and so I'm going to go here and I just need to make sure I have the folder so I think actually I can just do this rainbow slash rainbow and now I've changed the code and since I changed the code I also need to re-upload the code to the server so I'm overwriting it continue so now that new code is running on the server and I should be able to go back a directory so now what's here we have bot J s which triggers the processing application which is in rainbow and communicates with Twitter API in place to post the tweet so let's see if this works image bot is starting it worked and now if I go here and look at the bots we should see seven seconds ago this particular image now I want to make sure it's really over writing and making a new image so the only way I can think to really do that is to go into the processing sketch for a second and change this to like a green value so let's make whoops you can see now the color should be green and I'm going to export this export application as a Linux application go into here take this and I probably shouldn't be doing this right now I'm going to upload it so replacing it here we go so now I uploaded the new application and I'm going to go into that directory I'm going to whoops change the permissions again you don't ever need to do this I just want to really make sure it's working which is risky because it might not be I'm going to go back up a directory and I'm going to say node bot J s image bot is starting it says it works but I'm not entirely sure it over wrote that file I have a bad feeling it didn't I did okay that's impactful so it does work so and I can run it again now this particular bot by the way you notice it just quits it only does it once and you can see we got another one these are too even though there's a different color scheme they're different I should just make a processing sketch that makes a much more recognizable image but here's the thing what you want probably is your bot to be running continuously so the one thing is just to add set interval to the code but the other thing that you need is to use a tool known as for ever so this is the last piece that's on that's on this particular walkthrough so in other words what you want is for the node app to run forever in the background even when you log out of Amazon your instance and forever we saw that this nohup thing is all way of doing it which is also a way but no it has its own way of doing that which is something called forever so I can say sudo npm install forever - gee so this is a node module it's not associated with a particular node application but sort of global thing that you would run across a lot of applications so if I copy that here that's going to install this tool called forever and then it's going to my in forever install dance and then and then I can now start and stop things I can say forever start bot j/s which is running it and you know you can set all sorts of parameters like you know how long should it be up for how long should it sleep and all that thing but I'm just letting it use the default and another thing I can type now is forever dot list I think and you can see let's let's hold on let's make this a little bit more easy for you to see forever dot list and you can see that this is the particular process that's running nodejs bot j/s so if i wanted that was just running now forever it's not tweeting forever because the process is not set to do that but I could I can do things like say forever stop all I believe which just stopped everything so now if I say forever list again there's no forever processes running and I can say forever start again so there's lots of different ways so so this is what you want to do you want to you end so if I set that running and I log out my bot is now going to be running forever on that Amazon ec2 instance whoof this is like probably one of the longest tutorial videos I've ever made it was completely sort of like frantic animal haphazard this is probably hard to follow probably to be honest with you this page of instructions might actually make it easier but let me just review to recap what I covered this particular very long tutorial which is that first thing you want to do is create that server instance right you need to get a key pair a file so you can log into it once you've logged into it you want to SSH into it you can connect to it from terminal then you want to do things like install node then you want to connect to it with FTP SFTP cyberduck being a tool for doing that that allows you to upload your files there once you've uploaded your files there you can run your BOTS but if your bot also runs a processing sketch you want to also install Java run these commands to create the fake display in the background then export your processing sketch as a Linux application and upload that once you have all of that there then you can use forever to run your node bot that triggers the processing sketch this is like the the kick the cat that found on my father bought for two zeusie but anyway instead that that that runs the node that runs the processing sketch that creates the fake display that runs with forever my father bought for anyway whatever um so all those steps and you've got your thing running so I've anybody actually watched this entire video it actually has a node bot running on Amazon ec2 instance successfully I don't know please send me a tweet or something or to let me know because because that would make me happy to hear that okay so I'm going to stop recording now I will in the description for this video link to the written instructions link to all the references you need and hopefully this will be something that's useful to somebody someday somewhere in the universe

Channel: The Coding Train

Views: 30,671

Rating: 4.9724612 out of 5

Keywords: node.js, node, processing, nodejs, twitter, twitter bot, tutorial, amazon ec2, ec2, deploy

Id: 26bajyD4fLg

Length: 32min 59sec (1979 seconds)

Published: Tue Mar 22 2016