Linkedin Message Automation with Selenium

hello everybody today we will use our supreme python skills to automate mass messaging on linkedin now if you guys have already tried scraping linkedin you all know it's not an easy task however i am here to simplify it for you so that you will never have to pay for premium or email services ever again because my bot will do it for free for this tutorial we will use selenium once again but this time it will be much much quicker for this i've partnered up with tab 9 which is a really cool ai powered code completion tool and with it we will only type two or three characters and then tab 9 will suggest us the rest of the command it is of course available for all popular ides including pycharm and vs code and my favorite jupiter it has great privacy features and it covers many different programming languages not just python so if you guys want to find out more about tab 9 navigate to the link in the description of the video and sign up for a free account and when i say free i actually mean it it's free for life there's no strings attached that's why i'm so excited to share it with you guys okay let's fill that bot already alrighty so we will begin by installing tab 9 inside our jupyter notebook and while i'll be copying my commands from github i actually have them ready for you guys in the description so you don't need to go anywhere just copy and paste them and you will need to hit enter for the very last line and boom we have tab 9 installed and we can run our jupyter notebook and once we navigate to a folder of our choice we will start a brand new notebook by clicking on new and python 3. to save us some time we will actually copy all the module imports also from the description of the video and we will paste them in the first cell and run it with shift enter once we do that we can move on with activating tab 9. all you need to do is press shift and space and you got it our first command would be initializing our web driver and if you guys are not entirely familiar with webdriver where to download it or how to install it definitely check out my web scraping instagram with selenium video where i explain everything in detail and the link is also in the description so you don't need to go very far so our first command as i mentioned earlier would be web driver oh cool tab 9 already knows and right after the web driver command we specify the browser we would like to use in my case that's chrome and inside the round brackets we specify the url of our webdriver in my case it is saved in this wonderful folder which we will copy you see it's right over here and we will paste this url inside a string while switching all the backslashes to slashes and adding the name of the file which is chrome driver dot exe and lastly we will assign it to a variable called driver and this will basically initialize our automated browser window in the next line of code we will open the url we would like to request so let's type driver dot get and inside the round brackets we will specify https linkedin dot com and now if we run this cell with shift enter our browser window will open on the other side of my monitor for some reason but at least it opened in the right url address it's linkedin.com and we can start scraping now the first step with automating any social media actions is to log into our personal account to do this we will need to know exactly how those two input fields and button look like in terms of code for this we will open the developer tools with a right mouse click anywhere along the page and then clicking on inspect i'm going to enlarge it a bit to make sure everybody sees then we will select the arrow button and with the arrow button pressed we will click on one of the input fields and right away we can see that this is an input field with the name of session key if we do the exact same for the password input we can see that this is also an input field but this time the name equals to session password lastly if we check out the sign in button we are getting a button element where we can actually select it by two factors we have the sign in text which we can definitely target however what i'm going to target is the type attribute which has the type of submit and this is a very standard button across many different html forms that's why it makes a lot of sense to target it by its type and once we know the attributes of all the dome elements we would like to interact with it's time to select them inside our code we will begin with a username so we will type username equals driver dot find element by xpath and inside a set of round brackets we can specify the xpath of our element in our case that would be an input field with the name of session key and we will do the exact same thing for our password we'll just copy this line of code and we will refactor username to password and we will also copy password and paste it because the name of this input field is session password and not session key and once we selected those two inputs we can go ahead and fill in our personal information inside them we can do this easily by typing username dot send underscore keys and inside the round brackets in a string we specify our username and we'll do the same for the password we'll type password dot send underscore keys and this time highly confidential and last but not least we'll need to select the sign in button we can do this by copying one of these lines of code once again and refactoring to submit and instead of selecting an input field we'll select a button instead of targeting the name attribute will target the type attribute that equals submit and since the only interaction with this button will be a click event we can just add it to the end of the line we'll type dot click and then a set of brackets now one other thing that will make lots of sense is to fix my typos that could have been very embarrassing and one more thing that we didn't take into account is that every time we are loading a brand new web page we need to allow enough time for all the elements to load otherwise we might try targeting elements that are still being loaded and therefore they do not exist on a page just yet we can easily solve it by adding some weight in a few as strategic spots so we will do this right before we select our username and password we will type time dot sleep and we will specify two seconds we will copy this command we'll paste it right after typing our username and password we will paste it also after clicking on the submit button because it will also open a brand new web page and once we do that we can go ahead and click on sell and run all oh my god maria what is wrong with you today with all those typos okay now we need to rerun all the cells and boom we are logged in to our linkedin account and once we got that out of the way let's navigate to the linkedin page where we can message all our contacts through to do this we will just click on the search field we'll hit enter we will select people we will select our first order of connections and we got it you can see how easy it is to message each and every one of these contacts and not only this we actually have many many different pages of contacts in this tutorial we will only focus on the first page in the next one i will show you how to expand our scraper to target as many pages as you'd like so let's scroll up and take a look at one of these message elements we will open our developer tools once again we will select the arrow button once again and we will select one of these messages and even though it shows up as a span element we are much more interested in its parent button element and the reason why we focus on the button element and not on the span element is because you cannot interact with spans and that's why we will focus on its parent element which is clickable and which will trigger an action if we click it now it is also very easy to see that all the buttons we are interested at will have the exact same text aka message so what we'll do inside our code is we will target all the button elements with the text of message and before we minimize our window we will copy this url and we will paste it inside a driver.get command since we open a brand new page we will add some sleep so let's select all the buttons by typing all underscore buttons equals driver dot find element by tag name and it's actually elements it's not element and inside the round brackets we specify the tag name which is button but we don't want to click on all the buttons we only want to click on the message buttons that's why in the line underneath we will type message underscore buttons and this will equal to a list comprehension and inside this list comprehension we will only keep button for button in all buttons if and only if button dot text equals message actually let's make it message buttons and before we run our code let's try to click on the first message button so i'm going to copy message buttons in the index of zero and we will try to click on it let's see how it goes so we will click on cell and run all boom it worked and you guys can probably tell that i've already tested the scraper and it works like a charm my apologies jesse cool now let's have a look at the source code of this page once again and we will select this write a message text uh oh this is the most generic html element you can ever find this paragraph doesn't have a class name it doesn't have an id and it has no particular attributes that we can target how on earth are we supposed to send our message keys into it and this is actually the very first blocker we encounter now there's a very easy way to bypass it all we need to do is to select all the paragraphs on a page figure out which one of these paragraphs is the one we're looking for by its index and then sending our keys to the paragraph in the particular index we found we can do this by typing some sort of a text in here let's say gggg and if we expand this paragraph element we can see our text is already inside it so we are not going to close this window we will just minimize it and in the very next line of code we will type paragraphs equals driver find elements by tag name and in our case the tag name is p and in the next couple lines we'll figure out exactly which paragraph we need we can do this with a for loop so we'll type for p in paragraphs print p dot text and i'm actually going to add a counter variable we'll initialize it at 0 and this will help us keep track of the index so we will print it right above the text we will write print counter and in the very end of this for loop we will increment counter by one cool let's run this cell with shift enter and we see that we got lots and lots of different paragraph elements however our element is almost at the very very end of the list so i think the best way to go is instead of targeting index 38 in our paragraphs we will just target index minus five we'll just count the indices from the end of the list we can get rid of all these commands and instead of them we will type print paragraphs in the index of minus five actually the text of this element sorry guys and we got the correct text awesome so we know exactly which paragraph we're aiming for however if we'll try to send our keys to that paragraph at this stage we will encounter another blocker let me elaborate we will select this arrow button once again and this time we will select the div element who is the parent of our paragraph element it's actually this guy over here now keep a very close look at the class of this element and see what happens to this class as soon as i click on this paragraph boom what just happened we have just encountered our second blocker and the keynote here is to pay attention to the is active keyword which did not exist before we clicked on the div so in order to get access to our paragraph element we first need to click on its parent div element so what we'll do is we will copy the beginning of the class of this div we will minimize the browser window and right after we clicked on the button we can select our main div and we will do this again with xpath because xpath allows us to be a little bit ambiguous we don't have to specify the full class name we can specify how the class name begins or certain characters that the class name contains that's why xpath is the way to go we will type driver dot find element by xpath we are looking for a div element and inside the square brackets we will specify that this element starts with the class of the string we just copied from the developer tools and then right after we will select it we will also click on it which will then give us access to our paragraph element so we can copy this line of code can paste it right below and we will send our message to it by typing paragraphs in the index of minus five dot send underscore keys and at this point we will just test it so we'll type testing and we can obviously delete these two lines of code we don't need them anymore and just to be on the safe side we will copy this time sleep command and we will paste it right below clicking on our message button so are you guys ready let's close our browser window and let's rerun all these cells boom now let's see how we send these keys and how we minimize this div so we will open the developer tools as you can guess we will select the arrow button once again and we will see how our send button looks like ha so this looks like a copy paste situation because this button also has the type of submit and we've already done that so we are only left with this close button so let's have a look and as i mentioned earlier we are not interested in svg elements or in non-interactable elements we are interested at the button parent element and a good way to differentiate between this button and any other button on the page is the close conversation window attribute value so let's go ahead and copy this data control name along with its value we will minimize the page we will paste it as a comment right underneath and let's select this button with an xpath once again we will type close button equals driver dot find element by xpath and this time we are looking for a button element that starts with and we will copy the attribute from our comment we'll add an add character and paste it we will separate it with a comma and then inside a string we will paste the attribute value and after we do that of course we will click on that button so let's see if it works before we hit on submit actually let's just so we don't forget about it let's copy it from this line of code and we will paste it right before the close button we'll just keep it as a comment for now i'm just trying to minimize the amount of people i'll be pissing off with my scraper so let's go ahead and close our browser window and let's test if our close button works we'll click on cell run all and we have a typo situation once again the button that starts with not start with and if we rerun this once again boom our div just closed so we are almost ready let's turn our code into a for loop that targets each of these message buttons and not just the first one and as you guys can probably guess our loop will start here we will type for i in range from zero to the length of message buttons which is our list of buttons and now we will simply indent the rest of these lines of code with tab and we will replace index 0 with i which is our iteration variable that will loop from 0 up until the length of our message buttons but since i already messaged a bunch of people on that page i will avoid messaging all of them i'm just going to keep it to the last two people so let's say from index 8 up until the length of our message buttons which is 10. and instead of messaging them with the word testing i will actually copy a predefined message just so that people don't think i'm super weird i'm okay with being a little weird but super no no no no and obviously we will need to get rid of our submit button comment and to actually submit the message and actually just to be on the safe side i'm gonna add two more weights one of them is after we hit the submit button and the other one is after we close the div now we will go ahead and run all these cells and boom welcome to blocker number three so if we will close this browser window and we take a look inside our code we get the element not interactable exception and this exception doesn't mean that you've done anything wrong all it means is that the linkedin people think they can trick us and they can prevent us from automating processes just by raising this stupid error now you guys know me by now you know that it takes much more than this to prevent me from building my bots so every time you encounter this type of blocker make sure you are clicking on the element with javascript rather than python that's why we will dispose of this click command and on the very next line of code we will type driver dot execute script and as the first parameter we will pass our javascript code that would be arguments in the index of zero dot click and a semicolon while the second parameter would be the element we would like to click on in our case that's the close button and since linkedin has already tried tricking us we will know it might do this again so instead of using python for all our click commands we will use javascript let's do this and i forgot to wait one more time right after we are sending our keys to the browser so we will paste it here now let's go ahead and rerun all the cells sit back and enjoy no hands boom boom perfect so two of my linkedin friends think i'm super weird now great that's exactly what i wanted and that's pretty much it for part one so if you guys want to learn how to message multiple pages of contacts how to personalize each and every one of these messages and generally how to make your bots seem more human-like stay tuned for part two of the scraper and i shall teach you all that wisdom now thank you so much for watching and i will see you very soon

Channel: Python Simplified

Views: 24,367

Keywords: python, python programming, web scraping, web scrape, selenium, selenium webdriver, webdriver, tabnine, jupyter, jupyter notebook, python tutorial, python simplified, selenium python, web scraping with python, web scraping linkedin, linkedin bot, automation, linkedin automation, message automation, linkedin message automation, python bot, python projects, programming, coding, selenium automation, step by step, python programmer, bot, create bot, create a bot, programmer girl

Id: XdFUpFUDt88

Length: 23min 6sec (1386 seconds)

Published: Wed Apr 14 2021