Install Scrapy on Windows using Anaconda

hello so right now I'm going to show you how to install the scrappy from on a computer which is initially empty so we will start from 0 and we will end up running a few steps from the scrappy tutorial so this event has whether to get you started so we start with the Chrome and we go scrappy install windows typos are ok so we get here installation guide and as you can see it recommends and it's indeed a good idea to use anaconda so here we go download anaconda for Windows it's similar in Mac and Linux of course and we can use the Python 3.6 version this pop-up is relevant we can close it and we can see that the download started so this is important because it allows us to use python on windows that it comes with precompiled binaries otherwise we would have to compile from source and that would require Visual Studio and it's a much more complicated process so here we see our download takes some time in the meanwhile you might want to check and download your favorite text editor for example I like a lot notepad plus plus which is a simple one I guess you have your own favorite one so make sure you have a text editor because we will need that great icon to download completed as well so now just double clicking it will get us through the installation process next I agree just me default is fine install and this takes another couple of minutes so in the meanwhile it's a good time now to have a look on what packages are there included so actually you get a very nice environment with thousands of Python packages in our case we are Python 3.6 and 64-bit windows so here is the list you can see here beautiful super 4 which is quite nice scraping library as well contact Lib cryptography scraping this is also available and version 1.5 point all this is good you can see here psyche to learn this is very useful promising learning dance or flow again useful for machine learning requests this is also a very common library for writing web scrapers graph of these are very nice way to create nice graphs on diagrams by mc3 this is also great for machine learning Alex a lamb this is a useful library that is used from scrapey as well jupiter notebook this is very useful for experimenting and writing small scripts and visualizing key results and data graph is you can see here kit which is very useful when you develop for open source specially flask which is a very nice simple webserver bottle which is a library to work with Amazon Web Services and bottle 3 these are very useful specially when you are scraping and you want to upload files to s3 which is a thing that happens often returned just about time installation completes done so we click Next I think we can skip visuals to do for now okay closing this ones and we are done you might want to have a look on the tutorials they have but it's working for us from now so now we go back to the scrap installation guide let's now start by rhyming anaconda here I'm a conduct prompt so an important thing here is to understand where you are you are in this directory users mean so if you do for example from here see the desktop then you are in the desktop a directory so it's important for you to be familiar and know where how what you see in the terminal is called the terminal or command prompt for Windows corresponds to the directory structure so for example if you go to see directory then its users me and then in here its desktop so your desktop is just a directory and this is what you see here so for example if I right-click here and I make a new directory folder scraping okay then over here I can - dear and I can see this creepy here and it's the same on this directory now can do loose it is creepy and now I'm in this directory so there is a correspondence between what I see in the command prompt and then which directory among the file system okay so let's get back R to scrappy and let's go to scrapy tutorial now that we finished the installation we should be able to follow the steps so for example scrapie start project tutorial I pasted this one here and we see this error so this is because we didn't really feel as the installation after installing anaconda we have to do this condo install okay so let's run this command with the same way we can install other Python packages if we want especially if they are not already installed proceed yes okay so completed let's go back to the tutorial and let's try now scrape Easter Project tutorial nice so now you can see here the directory created inside the scrapy directory in your desktop and you can see more commands here CD tutorial scrape it and spider but we will ignore those instructions now and continue the tutorial and this tells us to create this file inside tutorial spiders tutorial spiders so we will now open our favorite text editor and go to this directory file save us and create the file code spider dot py so we can see it here nice so now we copy paste this and we save it great so now let's go back to command prompt see the tutorial in order to be in the directory where we have the spiders and let's do it here let's see what files they are there and we can see scrape it'll come fix so this is the right directory to be it's this one tutorials creepy that configure so now we're in the right directory and now we run this command has creepy crawl quotes nice so if you see here what just happened to new files got created here codes one and quotes two so let's open them with a notepad plus plus and we will see that the two pages got scraped exactly the tutorial instructed us to do so if we see what it does here it parses two web page is one and the second one and it saves the resulting a file with such a name so this exactly what they did which means that we'll have successfully gone through this first part of the tutorial which shows us that the installation went fine so we learned lots on this video we learned how to install anaconda how to install scrapey and how to go through the first steps of the tutorial follow the rest of the tutorial get more familiar with the command prompt and the corresponding directories and the files and also get used to your text editor because we will use those much more in the future while we run more complex scraping projects thank you and have fun

Channel: Dimitrios Kouzis-Loukas

Views: 25,742

Rating: 4.8823528 out of 5

Keywords: scrapy, anaconda, python 3, windows

Id: 5cstRCeFDkY

Length: 9min 44sec (584 seconds)

Published: Fri Jul 06 2018