Automated Google Drive Backups in Python

[Music] what is going on guys welcome back in today's video we're going to learn how to automate backups using the google drive api and python or in more general terms we're going to learn how to use the google drive api in python with the example of backing up files so first of all we need to install a couple of modules provided by google and for that we open up a command line on windows for example cmd on linux terminal on mac the terminal and we type pip install dash dash upgrade which gives us the latest version in case we already have some versions and we're going to install google dash api dash python dash client then also separated by a blank or by white space to indicate that we install a second module we're going to install google dash off dash http lib2 and as a third module google dash off dash o off lip so those are the three modules if you press enter it's going to install i'm not going to do that because i already have those installed but this is what you're going to need for today's video then before we get into the actual coding we need to set up the api keys the credentials the authentication the authorization in the google platform so for that we're going to need a google account so you can use your own google account your your major main google account you can create a test account here um the thing is the account that you use is going to be the account that is going to do the backup so if you want to have the backups at a certain account this is the account that you should use for this video um and what we're going to do here is we're going to login if we're going to click here on google apps let me just move this real quick so i'm not blocking this we're going to go to the apps here and we're going to scroll down to more from google and we're going to click on browse all products we're going to scroll down once more to for developers and we're going to click on google cloud alternatively you can also just google google cloud and you're going to get to the respective page you want to click on console here to get to the cloud platform and there what you need to do is you need to create a project now in my case i already have one um so access for bin because i deleted it so let me go to the projects page here um and we're going to go okay now i went back let me go back to the console what we want to do is we want to create api keys credentials for a respective for a project so what we're going to do now is we're going to um go here to apis and services and we're going to click on the oauth consent screen this is what we need to do first before we do the credentials now the thing here is that if you don't have a project you're going to have to create one so we're going to need a project an application in order to actually do something here and for that i'm going to click here on create project you can also select an existing one if you have one already for some reason we're going to call this now tutorial backup whatever um and we're going to just create and this is going to create a project and then we need credentials for that project and this is uh we're going to create an app we're going to create credentials to that app and we're going to use that in order to um access it with python now what we're going to click here is we want to click on external because we cannot use internal unless we are part um or unless we are a google workspace user uh so we're going to click on external we're going to create and we need to to basically provide some basic information here we're going to call this now tutorial backup amp now for a support email i'm going to use the neural g drive gmail.com here which is the test account for today's video and here i'm going to also say neuralgi drive gmail.com save and continue for the scopes we can actually ignore that because we're going to use the scopes um in the in the python script anyway so i don't think we need to do it here at least as far as i know um and for the test users we're just going to add ourselves so we're going to say neural g drive gmail.com [Music] add save and continue and that is actually it we can now go back to the dashboard and we can move on to the credentials so here we actually generate the client ids that we need or the client id that we need uh to actually access the application so we click here on create credentials and we want to have an oauth client id so we click on that and we click here on desktop app so the application types the desktop app and we're going to call the client whatever i don't know desktop backup client create um and now what you get here is the client id and the client secret now first of all i hope i'm going to censor this and if i'm not censoring this i'm going to delete it anyway so it doesn't really matter but what you need here is the json file so you click here on download json and you save that on your desktop or wherever you want to save it but you have to move it then to the development directory and we're going to call this credentials.json this is the thing that we're going to use to authenticate the python script to authorize the python script to do anything in the first place so we have this now on the desktop let's move this into the g drive directory here sorry and the basic structure of this project is we have a main python file we have this credentials.json file and we have some files that we want to backup in my case i created a directory called backup files and here i have a text here i have another text and here i have a picture that i drew in paint so nothing fancy here just a couple of files to back up and now we can get into the actual coding and i'm going to have to look at my second screen a couple of times because google has um the functions that they named and i don't know them uh off of my head so i have to to look into uh into the prepared code here so what we're going to import here first of all is we're going to import os and we're going to import os dot path this is not related to google this is core python um and after that we're going to say from auth dot transport dot request by the way most of this should be documented in the google drive api documentation i changed i changed the script a little bit to fit the backup purposes but the general imports the general access off or 2d api is documented in the google drive api documentation so here google auth transport request we're going to import request then we're going to say from google.of2 dot credentials we're going to import credentials this is going to be needed for the authentication from google underscore of oauth lib we're going to or dot flow we're going to import the installed app flow from google api client and you should have all of this by installing via pip so dot discovery import built from google api client dot errors import http error and then from google api client dot http import media file upload so those are essentially the imports and now we can get started with the actual um accessing and the first thing you want to do is you want to specify the scope what are you going to do with the script what are we going to do with the script and for that we create a constant list here scopes and we're going to add here https www.googleapis.com slash off slash drive so we're going to use drive because up until now we didn't specify that we're going to use drive we just specify that we're going to use the google cloud platform and we're going to use that uh in order to access the drive so you can add multiple scopes here you're just looking at the documentation or you google to find the scopes you need but essentially here you specify what you're going to do and every time you do that you have to refresh the token because we're going to save a token um but if you want to change the scope you need to also delete the token and create a new one so this is important um we're going to start with credentials being equal to none and then we're going to now look if we have such a token already now we don't have one in the beginning because we haven't created one but we're going to have one later on so once we are authenticated we're going to use the token to log in so we're going to say if os dot path dot exists um and we're going to check if the token dot json file exists already if it does what we're going to do is we're going to load the credentials so we're going to say the credentials are credentials dot from um from authorized user file so this is a token and this is going to be token.json [Music] with the scopes that we have defined here now let me just see barely blocking this there you go um so this is how we load the credentials when we are already authorized so we when we have a token already otherwise if we don't have that so if the creds is um or actually we don't have to do if not um it's none we're going to do if not credits so if we don't have credentials yet or if not the credits don't ballads so either we don't have creds or they're not valid so basically expired um or basically invalid for some reason we're going to say okay if we do have cred so this is um a little bit uh redundant here because we're asking if not kratz um and then we're asking if kratz basically this is when this happens here but uh we're going to do it like that so if we have credentials and the credentials are expired and we we basically have a refresh token so and credits dot refresh token what we're going to do then is we're going to say crets.refresh and we're going to pass a request in here so basically if the credentials are there so we were able to load it from the token json file but they are expired and we can refresh them we're going to do so by refreshing them um and otherwise if that is not the case we don't have credentials so what we're going to do is we're going to create a new flow and installed app flow and we're going to create this from a secrets file so the secrets files are actually a credentials file so we're going to pass here and i'm going to do this in a new line credentials dot json and the scopes there you go so we load we create the app flow from the credentials file and then we can say okay the credentials are flow dot run local server port equals zero like that um and of course what we're going to do no matter what happens here is we are going to create that token file so whether we created uh the the credentials the first time or we just refresh the token we're going to update the changes or create the token file so we're going to say with open token.json in writing mode as token token dot writes krets dot to json so this is basically just taking that object and putting it into a json file so that's it for the authorization and for the authentication what we're going to do now is we're going to get into the actual backing up of the files and for that we're going to make a try accept structure so we're going to say try something accept something and the exception is going to be in this case an http error that we imported as e and in case something happens we're just going to print error plus string of e there you go and the logic now is that we create a drive service so we create a service we try to access the drive api with our authorization so we're going to say service equals built which is what we imported up here built um and we're going to build here first of all drive version 3 and the credentials that we need to access that are credits and now what we're going to do is we're going to make a first request and we're going to get a response from that request the request is going to be is there a directory in drive that has a certain name and this is the name of our backup folder so for example i'm going to have a backup folder 2022 in my drive and if i have this i'm going to find it otherwise it's going to say there is no such folder so i'm going to say response here is service dot files dot list and here we pass now the query which is name equals and then single quotation marks backup folder 2022 and then single quotation mark again and then and the mind type so we want to make sure that this is not a file called backup folder 2022. we want to make sure this is actually a google apps folder so we're going to say okay the mime type and then in single quotations again is application slash vnd dot google minus apps dot folder and if i didn't make any mistake that should be essentially what characterizes a folder with a name backup folder 2022 and then we're going to say spaces equals drive so this will then give us the actual list of folders with the name backup folder 2022 in the google drive api and we need to execute that to get the response like that and our goal is now to see is there something or not so what we're going to do is we want to say okay if there is no such folder so if not response and the response is going to be a dictionary and what we want to have is we want to have the files so the actual things that found if not response files what we're going to do is we're going to create such a folder if there is no folder like that we're going to create one so we're going to say file metadata equals and then we're going to pass or create a dictionary here and it's going to have the name backup folder 2022 and the mime type basically same thing here application slash vnd dot google apps dot folder like that so this is the metadata and this metadata is now going to be used to create an actual file in this case folder not file so file is going to be service dot files dot create and the body of the file is going to be file metadata and the fields are going to be id and we're going to execute that to create such a folder and then we're going to get this folder um the folder id actually so the id of the folder to know which one it is because we're going to need that to put stuff in that folder we don't just want to create a folder we also want to put stuff in that folder so we need to identify it so the folder id is going to be file dot get id like that um so this is the case if the folder does not exist otherwise if it does exist we're going to do the following thing we're going to say folder id equals response fast because now we do have the file so we did find something and we want to get the first thing that we found and we want to get the id of that so in both cases we get the id in a different way here we create and then we get the id and here we get the id from the response directly since it found something and this folder is now what we're focusing on and all we need to do now is we want to iterate through the files that we have here in the backup files and we want to upload those into the folder so we're going to say for file in os list dear so list all the files in the directory and we're going to say okay the directory that we're looking at is backup files and the file metadata is going to be derived from the file now so the file metadata is going to be just name whatever the file name is and parents is going to be the folder id because this basically means put that file into the folder that we created or we found backup folder 2022 um and that's that so this is how we put that in there and we want to create a new media file upload so we want to say media equals media file upload and the name or or yeah actually the name is um an f string which is backup files slash file like that so this creates the media and the last thing we need to do is we need to create that file so file um actually maybe not file let's let's call this a different thing upload file equals service dot files dot create the body is the file metadata the media body is media and the fields are id and then we execute that and last but not least we can just print backed up file file there you go so that's that i think though that we're going to have a problem because i didn't enable the google drive api yet um [Music] oauth client was deleted i think i'm running an old main file so let me run this one here um yeah so the first thing that happens is when you run that you're going to be asked to actually um accept this so to actually choose an account to access this tutorial backup app with and i'm going to use neural g drive here i'm going to click on it and um basically it says google hasn't verified the app because it's our app it's not an official app that was very verified it doesn't really matter we click on continue um because we trust ourselves and then we continue here and you can see here the authentication flow has completed now depending on the scopes you choose you're going to have to agree on different things so this is why this prompt is there to basically tell you okay this applications one once this application wants to do x y z are you okay with that and if we extend the scope we're going to have more things that we have to agree to but essentially it says okay the authentication flow has completed you may close this window so we can go now into pycharm and you can see here that we got an error for uh 403 which is basically forbidden and the reason for that is access not configured drive api has not used not been used in this project so i can enable it by visiting that link and i'm going to do so i'm going to click on that and this is going to get us back into the cloud platform and here we have to enable the google drive api for this application otherwise this won't work so we enable this and i hope this is it let's see how long it takes there you go and now it should work so first of all let's run this again and see if we have some problems okay we have another problem here is it the same one all right seems like it just took a couple of minutes to be actually enabled now it works all we need to do is we need to run the script and then it's going to back up the files into the cloud and for that we can go into google drive you can see i don't have any files here i don't have any directories here also in the trash no files nothing and now i can go to point charm and run the script and it's going to be authorized immediately because we have the token and then you see backed up file backed up file backed up file and now i can go into drive and we see here the new directory backup folder 2022 and in here a couple of files you can see here the picture the text and so on now i can go ahead and delete those and i can also delete them from the trash here and uh we still have the directory so we still have the folder and i can go ahead now and run this one more time and you see backed up backed up backed up and the folder is still here with the new content and the good thing is that it didn't create another folder right so we can go ahead now and delete this and it's going to create a new one important thing you want to also delete it from the trash because otherwise it's going to find the trash um it's going to find this folder in your drive in the trash and it's going to upload there you don't want that there's a more complicated way to filter what it what is in the trash and whatnot but the best thing is to just delete it completely and then we can run it again and it's going to upload the files into the new directory backups 2022 there you go backup folder 2022 so this is how you automate your backups using the google drive api in python so that's it for today's video hope you enjoyed hope you learned something if so let me know by hitting the like button leaving a comment in the comment section down below and of course don't forget to subscribe to this channel and hit the notification bell to not miss a single future video for free other than that thank you much for watching see you next video and bye [Music] you

Channel: NeuralNine

Views: 23,935

Keywords: python drive, google drive, python, automated backups, backups, backup automation, google drive backups, python backups

Id: fkWM7A-MxR0

Length: 23min 52sec (1432 seconds)

Published: Mon Mar 14 2022