Hide Your Identity | Tor using Python | Python for Beginners 2020

hey guys how's it going so in today's video i'm going to be showing you how to build a api that you can leverage but more importantly if you want to do things like web scraping for example and you're afraid that maybe your ip may get banned i'm actually going to show you how to use a socks proxy more importantly you're going to be navigating through the tor network to be pulling these api requests or these web scraping requests so in today's video we're going to do a couple of things first i'm going to show you how to actually run the proxy itself second every single time that we run the proxy we're going to go ahead and change the ip address so that we don't get banned by one specific ip and then last we're also going to put in a little check in here to make sure that we're actually protected and we're not going to be using our real ip a real public ip that we are in fact going to be using a proxy so i'm going to show you how to do that what we're going to be pulling is when i go ahead and run this you'll see what we're going to pull is we're going to pull the weather forecast except there's one thing missing here and that is i need to go ahead and start the tor network so to do that i'm just going to type in tor and we're going to walk through all this in in a very quick detail here but as soon as i run the tor proxy and i run this again because it's running through torts a little bit delayed but basically i'm fetching all of this data off of an open api through the tor network so i'm actually getting going through the tor proxy going through the onion router or the tor network which basically goes to three different layers so that it's very very difficult to trace back the original ip now it's not bulletproof but it's still pretty good so i'm going to show you exactly what this code does and how to build it so let's get started alright so let's talk through how this works so you have a client machine which is your device in this case and let's say your device has an ip1234 what's going to happen is that ip address is going to be sent across the actual headers and in addition to that you're also going to be sending some parameters so you're going to send for example i want the city to be equal to toronto i want the number of days to be equal to five and then you're going to be sending your key across as well and you're also sending other things in your header as well so you're sending the ip for example you're going to be sending things like the user agent and then also maybe some more device specific stuff so some more device information whatever that may be i apologize for the bad writing i'm trying my best on my ipad here to do this but anyways regardless that information then gets relayed over to the server and so the server is going to go and it's going to say okay i'm going to look into my database repository here and i'm going to go ahead and based on what you're looking for i'm going to go ahead and pull all that information based on these request headers or the parameters i'm going to take that information curate it i'm going to send it back as a response object and in the response object i'm basically going to have all my information so that's going to be equal to my forecast for my weather forecast and a whole bunch of other stuff and that gets displayed back to the client machine now the inherent problem behind this is when you're doing something like web scraping for example because your ip address is actually getting ported over and sent to the request headers the server for example now has access to that ip address with that ip address now they know that it's one two three four they can go back and triangulate your location as well but more importantly they also take your ip address and they'll go ahead and store it in their database and what they're going to look for is they're going to say if this person is excessively accessing the server for whatever reason whether i'm web scraping or doing whatever i'm going to go ahead and i'm simply going to block this ip address and i'm going to block this ip address i know who they are i know where the request is coming from and i'm just going to stop them from getting it now if you're doing web scraping and you're doing an excessive amount of it you probably shouldn't be unless you've actually entered some kind of agreement with them but let's just say you're not and you're web scraping i don't know whatever it may be job descriptions or maybe web scripting other other weather information that's not accessible through an api so you're using web scraping to do it you may be blocking yourself in your ip but at the same time you're also giving up certain data about yourself especially your location and whereabouts so to counter that what you do is you actually go ahead and you set up a proxy so again client machine is going to say i want to go one two three four i'm going to send that in my request header and then i'm also going to have some parameters again day is equal to five city is equal to toronto what happens in this case though is this information then gets goes through this proxy so what the proxy is going to do is if i've defined things like a fake user agent which we did in our code and we'll walk through in a very quick second and if i divide and i've also put in things like device specific info like i'm using a mac or an apple or whatever that is all that's going to get controlled by the proxy and what the proxy is going to do then the proxy is going to go ahead and say okay i'm going to send this information out but i'm going to use my own ip address to send it so i'm going to send it through 5 6 7 eight and this proxy is associated with some kind of a tor address and i can't go back and triangulate or figure out the location of the actual client machine because i've covered it up through a proxy now what happens is the same information happens here they're gonna go through their database they're going to go ahead and log certain information so they'll log this ip address they'll log your request headers then they'll go back send a response the response object goes back to the client great so what's the problem with this one although it gives you a little bit more anonymity the challenge here is that you don't have a floating ip address and what that means is every time i'm accessing the tor proxy i'm always going to get this ip address as a result so what can the server do the server can go ahead and block this ip address so while there's important information about you that is preserved and that is your location your ip your actual real ip address which is here as long as that is not disclosed you're okay but if you want to do continuous web scraping against it well now this proxy has been blocked so there's another way to counteract that and the way to do that is we're going to use something called a proxy control signal and what that does is this as follows so it's the exact same thing we're going to say location is equal to toronto days is equal to five this is going to say ip1 one two three four that's gonna go ahead and get sent to the proxy signal it's gonna kick out five six seven eight okay and it's gonna have the exact same parameters so it's gonna use those parameters send those through the user agent all that other kind of good stuff is gonna get sent to the server so we have the user agent here okay um what's gonna happen here is it's going to go back send the information curate the information here send it back their response and go back okay but now going back like the previous slide somebody can say well you're going to ban five six seven eight here's the beauty every single time you run this command and you say hey listen i want to actually go ahead and get something this ip address is a floating ip address which means that it's going to change every single time so what's going to happen is instead of it going into 5 6 7 8 the next time it's going to go into 9 10 11 12. and then the next time you run it it's going to go into and i mean these numbers are completely making up 100.1.1.2 or whatever it is this number is going to change so the server is never going to get to know a consistent pattern of all the different ip addresses that are going through and so what you're doing is you're protecting your original ip address here you're getting the information you want you're changing around sort of disguising your user agent and stuff like that it's going to go ahead and take all this information and it's going to you know disguise it then it's going to send it to the server and this ip address changes and because even if they log this ip address it doesn't make a difference because the next time they run it it's going to change so this isn't concept how the control proxy signal works we're going to build this out in python and i'm going to show you how to do it so let's get started okay so obviously the first thing you're going to need to do is you need to go and install tor and the best way to do that is just type in brew install tor in your command line so go ahead and open up a terminal window for example and we'll just do that right here so we're going to bring in this terminal window and you're just going to type in brew install tor now i already have it so i'm not going to run it the minute you do that it's going to take it takes depending on your computer it could take a couple of minutes for that to run once that's actually done then you can go ahead and start the tor network just by punching in tor and you're going to notice that it's going to go through all this bootstrapping and then finally it's going to say done once it says done you're going to be already on the tor network now to get tor set up especially on mac os there's also some instructions for windows which i'll link down below but for mac os there's a few things you're going to need to do to get this running so getting tor running on its own this should be enough but if you actually want to get your ip changed every single time you run it so that you're not running into the same ip address there's something that we're going to have to go ahead and enable so in order for us to do that i'm going to hit and open up a finder window and we're going to type in user slash local slash etc tor slash t-o-r-r-c and that basically opens up the tor folder you're not going to have this file what you're going to have is you're going to have something called tor rc.sample and all you're going to do is you're going to copy and paste this and you're going to remove the dot sample so you're just going to have a duplicate copy of this once you have that there's only a few things we need to change we're going to scroll down and you're going to look for a couple of lines that we're going to uncomment all that does is it just allows us to go ahead and use something called control port to get it get a new ip address every time so this is going to be commented like this what you're going to do is you're going to uncomment this one this one's also going to be commented this hash control password and then the cookie authentication now this password here in itself even though you see that it's grayed out or it's blocked out here you can go ahead and change this so even if it is exposed it's not a big issue but anyways and then uncommon cookie authentication and that's all we need to do with our torrc file so we're going to go ahead and close that i'm not going to save it because i already have everything so now in order to get this running we're going to go back to our tor network here and i'm going to show you a couple of things okay so first let's go ahead and bring in some dependencies so i'm going to bring in this we're just going to file save this and i'll just call this put it on my desktop or something and i'll just call this i don't know tor example dot pi so we'll call it tor ex dot pi or something like that okay and then i'll just put it on my desktop and we're good to go so hit save good done and that's all i really needed what i'm going to do is i'm actually going to show you how these proxies work so we're going to go and print out some actual proxy names or some proxy ips so you're going to see that there's my public ip now my public ip obviously i'm not going to show here what i'm going to do is i'm going to log into my vpn provider which in this case is nord if you guys do want to check them out feel free to do so i'll leave a link down below but we're going to go ahead and log into nord and what nord is going to act is nord is going to act as my my public ip for now and the reason why i'm doing that is just because i don't want to expose my real public ip you should never do that always keep your public ip private to yourselves but let me go ahead and get an ip address and then i'm going to quickly connect and then i'll show you that we're going to be connecting into here so what you're going to see all right so now that i'm logged in what you're going to see i have an ip address here so when i click over there it's 104.206.26.139 now that's going to serve as my public ip right now so we're going to go ahead and actually run that in in python so that you can actually see that it returns that back as my public ip so let me go ahead and get this script here or this a little bit of code here that i put together so basically it's a request that we're going to do into this website called http bin so we're just going to go ahead and do this i'm going to say request.get and that's the text file that we're going to be looking to get and i'm just making sure i don't have anything else that i need and then we're just going to print this out now what this should return is this should return that exact same ip address so when i do that it's 104.206 i think that was the same thing that we had here so 104.206.206.139. if i go ahead and change this so just to prove to you that this is actually pulling whatever is in nord which is acting as my public ip at this point so if i go down and i don't know let's just pick something in the states and i run that again i get 91.132.137.100. let's pull that so that's the exact same address again so again this is helping you understand that assuming that i didn't have a vpn on this would actually display my real ip so every single time you send a request over to whatever website you're going to be doing they will log your ip and what happens is once they log this ip if you pull it too many times or you break their terms and conditions they will ban the ip so they'll just basically blacklist your ip and you won't be able to pull anything anymore and especially if you're on your public ip and you're not proficient in setting up vpns and proxies you might run yourself into a little bit of trouble of not being able to do any more data gathering so what we're going to do now is we're going to go ahead and set up a tor proxy before i even go ahead and set up the ability to change your proxy multiple times i'm going to go ahead and just set up a tour proxy on its own so what we're going to do is we're going to create a new variable called session now requests has sessions built into it and that allows you to open up a specific session under the actual request module so we're going to define a variable called session now under session i'm also going to define something called proxies so for that we're going to have session.proxies and the two proxies that are for tor are actually sox5127001 so this is your local host basically at port 9050 and then also if it's an https connection exact same scenario here as well what you're going to see now is i'm going to leave this here actually and i'm just going to copy and paste this and now we're actually going to say sessions and what this is going to do is it's going to say okay i'm going to open up a new request session and i'm going to use the proxy to go ahead and pull this url here so let's see what we get back when we run this now it's actually saying there's no connection that's because again i need to go ahead and run the tor network so let's go ahead and run tor so you see that this is now saying done that's what we need now when i run this again it's saying origin this is my original proxy which is my nord proxy again acting as my public ip and this is the proxy that the tor network has given me and so no matter how many times i run this i'm going to get the exact same result because right now i haven't told it to go ahead and change and because i haven't told it to go ahead and change technically what they can do is even though this is acting as my proxy right now they can go ahead and ban this ip if they wanted to and in which case my session variable or my session for this specific tor proxy is no longer going to work as well unless i go ahead and change it now the very simple way to change it is you would just go ahead go to your terminal file you'd go ahead and terminate this this is one way of doing it and rerun tor and you'll notice that when i run this just note this one eight five one nine two dot two one two that's actually going to change into something completely different so this now ends in two four nine and before it ended in two one zero i think or something like that and you can go ahead and always reboot or you know refresh your ip address that way but there's a better way to do it and that's especially when you want to do multiple pulls at one specific time you want to automate things end to end your data extraction needs to get automated so in order for us to do that then we need to go ahead and introduce something called a controller and the controller basically what it does so if i go ahead and just bring this snippet of code right here okay so what the controller is going to do is it's going to say okay every single time i run this function called new tor id which is what i basically called it i'm saying that on port 9051 i'm going to run i'm going to send basically another signal for me to go ahead and get a brand new ip address so in order for us to clean this up a bit let me just add in a few other things in order for controller to work we need to go ahead and add in a few other libraries as well and they include stem which is for the signal and then for the controller as well and this basically allows you to go ahead and authenticate so if you go back to your original file uh the the tor rc file there is that long password that i was talking about that's sort of what this is authenticating against and you can go ahead and change that at any point in time as well as long as the two match both in here and as well as over there right now it's just pulling it from the tor rc so it shouldn't be an issue all right so now that i've actually gone ahead and done that i'm actually going to just go ahead and import the other libraries as well these are very basic ones there's nothing uh crazy about these ones here so p print is just to make sure your json looks clean when you print it i'm going to need json to convert a string file into a json and then also i'm going to need the os to go ahead and pull an environment variable because i didn't want to put my api key up here i'd rather just pull it put it into the environment variable which you guys can look up how to do but let's go ahead and clean this code up a little bit so now here's what i'm going to do i'm going to replace this piece here well that says all this over here basically getting rid of all of this and we're putting in this code so let's walk through what this is so this is the same as before i'm getting a session uh method basically from my requests function here okay so the next thing we're doing is we're going to go ahead and again set up the same proxy command that we had previously the other thing i've introduced here is actually something called session.headers and when i'm passing on my headers i want to disguise my user agent so every single time change up the user agent whether it's firefox whether it's safari whether it's a mac whether it's something else i would rather just disguise that so they don't that they can't figure out that it's coming from one specific device and then for this specific api we're gonna also gonna pass some parameters so q in this case i'm just pulling in toronto the key is going to be the weather api so what we're doing in this we're pulling the weather current forecast but also the forecast for the next x number of days and we can define that here whether it's one day's five days whatever it is and then we're going to get a whole bunch of data so i haven't defined the url piece yet but why don't we do that now before i go ahead so let me put in this here and i'm going to show you what happens with this as well but this is basically calling this function that's saying that every single time that i'm going to be pulling this api i want you to go and give me a new id and what i'm going to demonstrate to you now is the fact that it is actually going to pull in uip every single time so let's forget about the top one because we know that's my nord one but let's go ahead and pull this and so it's giving me an idea of one four six five nine one five five two seven you generally have to wait about five to ten seconds but then when we pull another one let's see if it's refreshed yet or not there it is one eight five two two zero so it's a completely new ip than what i originally had before and these are all new so i'm gonna run it again and you notice i get another ip address now if i were to go ahead and just block this function and comment it out my ip's not going to change no matter how many times so let's just stick with this one so it ends in 201 so i can run this as many times as i want i'm not going to get a brand new ip address because i basically told it don't go ahead and send a signal for it to go and renew the ip address i'm just going to keep the same one and that is kind of the beauty of way you know the way this function works the other way i can do this is if i could just go ahead and so i get a new id there but i can block it at the proxy level so i can say i'm just going to turn off the proxy up here and this again is not going to change again so it'll change the one time because because i had already ran before but now if i keep running it over and over again it doesn't matter how many times i run it i'm going to keep getting the exact same ip address and then when i unblock this it runs into a brand new ip address again so that's kind of the beauty behind how this works with the tor network so i'm just refreshing away and you see that i'm getting a new ip address every now and then and that is what you want to see so when i go ahead and pull this api which is going to give me a whole bunch of data and so let's just go ahead and run this so let's see what this api is going to give me so let's call in the url so the url for this api let's get rid of this for a sec so the url for this is basically http api.weatherapi.com and then i'm going to go ahead and use my session variable so i'm going to use the proxy to go ahead and get this url in there i'm passing in the fact that i'm using the tor proxy i'm disguising my user agent and these are the different parameters that i'm going to be passing and then i've gone ahead and done the json loads on this because this returns a text file here and what you want to do is you want to convert it into a json so that i can use p print to to make it look a lot better so when i go ahead and run this what you'll see now is there's just a whole bunch of data here so it gives me the current condition um it'll also give me the forecast by hour so there's a forecast there it'll give me for today's date it'll give you for different times of the day i believe as well which is why there's so many different um lines here but you can see how it's it's very well structured and i'm going to actually turn off p print in just a second to show you that if i don't use p print it's just almost impossible to read this stuff but there's a whole bunch of data in here that you can sort of sift through and look at i'm just trying to find where that hour number is because i know it's in here for sure but if you scroll through this you'll see that it actually does show that it's at specific times of the day but regardless um you know this is how you would pull this kind of information so if i turn off p print and i just use regular print this is what i'm going to get so not very usable this is why you have to convert it and use the json loads to convert it into a json and then go ahead and read it and i think it outputs this as a dictionary anyway so that's kind of how you would pull this api and then you can use this data you can ingest it in whatever mechanism you want you can slice and dice it and do whatever you want but there's one more thing i want to show you and that is sometimes you may forget to actually run the tor network so sometimes you may just you know not realize that you haven't read it or for whatever reason you may have something like this commented out the proxy and not realize that it's commented out because you're not going to necessarily always run it from here you may actually be running it from the terminal so you may not even see the code so now we're going to put in a little bit of logic okay and here's the logic so i'm going to change these lines up a little bit and the logic is going to be like so so let me just go ahead and grab this okay so we're just going to come over here and i'm going to copy paste you're going to see this exact same code so this isn't going to change you're just going to see that it's down here so the logic that i put in as i said if i use the request stock yet which is going to give me my private ip or my public ip the only one that i should know and if i run that ip address and it's the exact same thing as what the session variable gets me then that tells me my proxy is not on and what i mean by that is that means that my two ip addresses are the exact same my public ip address and then my proxied ip address are exactly the same and the reason they're the same as the proxy isn't on so when you run something like that in this scenario it's going to work because i know that they're not the same so when i run this it's going to take a little bit of time but it runs it but if i actually go ahead and turn off my proxy so let's go up here and actually shut off my proxy so that i'm just passing through my raw ip address and run this it's going to say unprotected the proxy is off because it's doing this check and it's saying your proxy is actually not on this thing is returning the exact same thing as this thing so you're actually putting yourself at risk but the minute i turn my proxy back on and rerun it there you go so that's basically how you run it and now this could be your request url here you could be using an api you could be using something like web scraping because at the same time if you use something like beautiful soup you're going to be using the request library anyways so many different things you can use this for but the but the most important thing is you protect yourself and protect your ip address but i would say even if the even if the website allows you to web scrape and i generally look at the robots.txt and sort of obey the rules and follow the rules for the for from the website anyways i would still proxy it because i don't want my own private ip floating around out there there's no need for it there's not anything they're gonna get out of it um so if i go ahead and just you know disguise this no harm no foul nothing nothing goes wrong i'm not doing anything wrong neither should you you shouldn't be using this for the wrong intent or purposes because like i said this isn't 100 bulletproof there's other ways to add on to this um which i'm not going to discuss here in this video but maybe in another one to further secure yourself even more but other than that tor is a really good way to secure yourself it's a really good network the challenge and the and the danger with tor is really depending on who's sitting on the sort of the end node if it's somebody who's not who doesn't have the best intentions they can they can steal some very valuable information and if you spend enough time and you gather gather the time stamps you can sort of figure out a lot in terms of where the request came from the the middle node as well as the the endnote as well or the exit note as they call it on tour so it's not always bulletproof um and for that reason i'd encourage you always to use it for good purposes don't use it for the wrong intentions you can get yourself in trouble but outside of that i hope you found this educational i always use tor or i use some kind of a proxy paid mostly i don't really use public proxies but i've paid proxies as well that i'll use um but again only for good purposes but anyways hopefully you guys enjoyed this if you did please consider liking and subscribing and i will see you next time thanks very much

Channel: SATSifaction

Views: 1,508

Rating: 4.9333334 out of 5

Keywords: python, python for beginners, hide your identity online, vpn, tor, hacking, cyber security, guide, anonymous, tutorial, security, tor browser, internet, hackers, computer, video, anonymity, tails, hacker, windows, identity, privacy, dark web, linux, google, anon, design, cyber, howto, os, operating system, tech insider, ti, tech, science, innovation, digital culture, internet safety, how to become anonymous, identity theft, online, go incognito, techlore, tails os, deep web, best vpn, email, hide, learn python

Id: h7ibO9qUnNQ

Length: 25min 41sec (1541 seconds)

Published: Thu Oct 15 2020