Upload Large Files To OneDrive Using Microsoft Graph API In Python

hey howson guys in this video we're going to learn how to upload large files to onedrive with microsoft graph api in python all right so i want to dive into the documentation first so to upload a large file to onedrive it's a two-step process using the api the first phase is going to be we're going to create upload session and the second phase is going to be uploading the file to the upload session and which is going to be like a container so i tried these four endpoints and apparently only two endpoints are working and i'll show you which endpoints are working as for the permission scopes we can use the files.rewritescope rewrite.oscope in the sites.rewrite.oscope and here's an example of the request body yeah let's see right so that's pretty much it if you look at the example unfortunately the example only show you the first portion of the step which is uploading the first chunk and since it will be uploading a large file that means that there will be multiple chunks the documentation then showed the remaining steps unfortunately all right so that's pretty much it oh i guess that's going to be all okay so let's dive into the tutorial and before we dive into the script make sure that you download the source code to this ms graph.pi file all right so in my script i'm going to import the os module and increase the font size all right so i'll be using this report.pbx file as example all right so from the import statement i'm going to import dos module and to make an api code i'm going to import the requests library and from ms graph i'm going to import the generate access token function in the graph api endpoint url all right so i did insert my app id i'm going to create my scopes i'm going to use the files right scope all right so for step one going to generate the access token and here i'm going to insert the generate access token function and i'll provide my app id in the scopes and this will be on generate once i obtain the access token i can create my headers option and it's going to be authorization bill space and the assets token all right so let me go ahead and run this code block to obtain the access token as for the file so i'm going to grab the file path and let's go to uh step two and it's going to be step 2.1 actually last name this is step 2.1 and step 2.2 all right so here i'm going to create two variables one variable is going to be the uh full file path and the file name is going to be report that it should be reported to the pbix let me just double check oh there's no two in front of the file path i want to extract the file name using the oyster path database name function and here i'm going to insert a routine to check if the file exists so i'm going to say if not os.path dot existence inside the function i'm going to insert a file path and if the file is not found onto risk exception and for the message i'm going to insert the file name it's not found otherwise i'm going to load the file and i'll name this as upload and you want to extract the by string of the file and i'll name the output content media content and from upload diary uh let's see actually let's make this step two read the file and it's going to be step 3. now if we go back to the documentation and if we look at the request body so here we have the option to include the metadata of the file we can include things like description and if the file it exists then how do you want to deal with the issue and we need to attach the request body to the item field all right so let's do this i'm going to create my class body uh just an example all right so inside the dictionary the field name is going to be item and for this demonstration i'm going to provide a description to the file and for the description i'm going to say a large upload file actually let's do a large file and we need to attach the file name that's going to be everything i'm going to provide and for the file i want to upload the file to my lessons folder so in the url i want to grab the folders id so it's going to be the value of this id property let me create the variable right so when i say upload large file so depending on your internet speed i have pretty good success with files less than 400 megabytes but if a file size is greater than 400 megabytes then we need to split the file into multiple chunks and the difference between step 3.1 and step 3.2 is step 3.1 requires one single session first start 3.2 requires multiple sessions to upload the file onto our onedrive's server all right so here i'm going to make a post request and it's going to be oh actually uh let's do this make sure that's fine and it's going to be create an upload session and the end point is going to be graph api endpoint and let's go back to the documentation so from the endpoint we need to use the second endpoint which is going to be this one here and since i'm uploading a file to my own onedrive account here i'm going to place drives with me and i'll change this to drive items followed by the folder id followed by the file name then we need to attach the headers and for the request body we're going to assign the addition to json's parameter and i'll name the outputs response upload session all right so let me go ahead and run this code block oh file name is not created let me take a look okay now let me rerun the code block right so if i print response upload session and should be json the response is going to return couple things the timestamp when the session is going to be expired and the file chunk wrench and the upload url so here i'm going to use the tri except block from the response upload session object i want to reference the json output they want to grab the upload url i'll name the output as upload url so basically this url was kind of like a temporary uh storage and we're going to upload our file to distance storage and d1 drive server is going to recognize the file and transfer the file from their server to our own onedrive account or onedrive folder now i need to make another class call and it's going to be a put request and i'm going to end the destination it's going to be upload url and to upload the file we're going to assign data with media content and i'll name the output response upload status and once the upload is complete i want to print the file path in the upload status everyone into any exception they want to put in the message so let me create a new file and i'll name the file demo2 right so let me go ahead and the terminated session now put a window in the script side by side now if i run the script so here the file path and the upload status all right so if i refresh my page and here's the report file now let me delete this file now i want to go to the second example which is going to be upload relodge file and i'm going to grab the code block from uh see let me delete this line first from line 39 to the import statement all right so if you want to upload a re-launch file basically our file die is larger than 300 or 400 megabytes depending on your internet speed and the upload process is going to be pretty similar except that here we need to chunk the file into multiple pieces all right so i'm going to grab the context manager actually let's grab this four lines and i'll move that to step 3.2 to here and i'll move the variable to the top all right so here from the context manager let me delete this line here i'm going to start everything from scratch i need to grab the file size first and i'll name this as total file size now i can grab the file size using the oyster path dot git size function now insert the file path and for the chunk size here let's go back to the documentation all right so right here for the chunk size it needs to be 227 680 bytes per chunk size and this is the requirement set by onedrive api so i'm going to copy the chunk size and i'll name that as chunk size then we need to figure out how many sessions are required to upload the file and we can get that number by referencing total file size divided by chunk size well the remainder and because each file size is going to be different so we're going to get a left over from the division operation the last chunk size of the last section and it's going to be total file size minus chunk size times chunk number then i need to create a counter to track the session upload all right so here i'm going to insert while loop i'm going to say while true and we're going to grab the by string of the first chunk and let's name this as chunk data then i'm going to grab the beginning index and it's going to be counter times chunk size and the end index and it's going to be star index plus the chunk size then we're going to insert an if condition if not chunk they want to break the operation and also if the counter number is equals to chunk number then we know this is going to be the last session then we're going to grab the style index plus the chunk left folder value now let's prepare for the classical so first we need to create a header software inside the headers object we need to specify the content length and it's going to be the chunk size then we need to specify the content range and it's going to be the by strings location and we also need to specify the total size of the file assurance do this i'll make this as f string and this will be star index and it's going to be end index and we need to subtract the value by one and this will be total file size now we can make a request code to upload the file and it's going to be requested output upload url okay what's my uh let me take a look the vehicle should go here and i also need to grab this code block and which i forgot is all right session yup that should be right and we don't need this line here the upload url is the only thing we need and here we insert the exception block and for this i'm going to just print the end message and human name diapers chunk data upload status and i'm going to assign chunk data to the data parameter and i'll put the headers here all right so let's do this i'm going to put a message upload progress that format and the value is going to be chunk data upload status dot json and from the json response i want to reference next expected ranges key that can increase the counter by one and just in case you want to cancel the session you can create another separate python script and you can make a delete request and you will insert the upload url to cancel the upload session all right so that's going to be dean high script and it's a bit more complicated than the first example right so let me run the script just for the test run oh so if you have a typo all right so here i got a key here i can see that the file is uploaded and here's the file now let me fix the script real quick right so i'm going to do this here let me print the json output from the chunk data upload status and i know when the file upload is complete we're going to get this uh create by keyword so i'm going to insert try block here and i'm going to check if this keyword created by using the json response they want to print the file metadata response i should also do file upload complete otherwise we're going to print the file upload progress for status and that should be it all right so to do one more testing i'm going to use one png file for the testing all right so here i get the message file upload complete now just in case if i go back to my onedrive and here's the file and the temp stem is less than a minute ago which is the this upload session that i just ran all right so this is going to be everything i'm going to cover in this video and hopefully you guys found this video useful and as always thank you guys for watching i'll see you guys in the next video

Channel: Jie Jenn

Views: 13,507

Keywords: Microsoft Graph API, OneDrive API, Automate OneDrive, Upload File To OneDrive API, OneDrive Python, Upload Large File To OneDrive API

Id: nSfHrNarqD0

Length: 25min 20sec (1520 seconds)

Published: Wed Apr 20 2022