How to Upload, Process, and Download CSV Files in Flask apps

hey everyone in today's video i want to show you how to upload and process csv files in a flask app and also how to download some files from a flask app some csv files so this is something that a lot of people have come to me for help with just how to handle csv files like how to go through the upload process so i thought i'd make a video and if you need any one-on-one help with this yourself i have a coaching program just go to prettyprinter.com coaching or the link in the description below where you can learn more on how you can talk to me and i can help you with your app so let's get into this one the first thing i want to show you is this script that i have so this script is like some kind of process that you would have to take in a csv file and do something in my particular case all it's doing is summing up some product types so here i have a product and price and for some of these products they have types so for example here we have a pepper and it's a chili powder pepper right and then we have garlic here so the point of this script here is just to remove everything before the hyphen and call it the product type if there's no hyphen then the product itself is the product type and then it sums up all the prices and then it returns a new pdf that has basically the product types and then the sum of all the prices so the way this works is i can go into interactive mode with my script and then i can call process csv with productdata.csv so product data.csv run it and then it's going to create a new output file that appends the timestamp i'll open it up and then this is what the file looks like so this is just an example process this is not like anything crazy probably a better way of doing this but this is my example so what i want to do is i want to build a flask app around this so right now it's just a script so what i'll do is i'll create a new app.pi file and i'll open that up and in here i'll do the typical flask import so from flask import flask and then what i'll do is i'll create a create app function and because my example is so short i'll keep everything in the create app function but if your app or larger you might want to organize things into multiple files so inside of create app i'll instantiate the flask object using dunder name and then i will return this object return app okay so the first thing i want to do is i want to create an upload form so i can allow the user to upload a csv file that needs to be processed so to do this i'll create a routes and we'll just call this upload a simple name and this will have both git and post requests and i'll just call this upload as a function and notice i'm putting this inside of the create app so it works the same if it were outside but just to keep everything in one file i'm putting it inside of this function create app here so when it comes to routes like this where there's both git and posts you have to check the method first so let me import a request from flask now also import render template because i'll need that in a moment so for the route the first thing i'll do is request method is posts right so for the post method i'll do one thing and then for the get method i'll just put this outside of the if block and i'll return render template and i want to return a template called upload.html which i'll create in a moment and then in the post request i'll actually take the file and then you know pass it to the process so it can run and then here i'm going to return a redirect so return redirect and then url4 so i need to import those at the top and then return redirect url for and what i'll do is i'll return to a download page that i don't have yet but i'll i'll return there once i have that working so what i want to do now is i want to create an upload.html file so what i'll do is i'll create a templates directory and i'll create a file in that templates directory called upload.html let's go ahead and open that and i'll keep this very simple so i won't put much of the html i'll just put the form itself and on the form i need to specify the action so the action is going to be upload i'll just make it explicit you know you can leave action as blank and it will just automatically post all the forms to the same endpoint you can put upload like this which what i'll do or you can use url for the method is going to be post and then finally when you have a form where you can upload files you have to put an encoding type so encoding type is equal to multipart slash form data just like that so a common issue is you can't get the file data when you upload it and it's probably because you're missing this here so now what i want to do is i just want to create the field for a file so what i'll do is i'll have a label and i'll say this label is for file and we'll say file and then we'll say file here and then we have the input type file id is going to be file and name is going to be file and then let's just put a line break here and then we'll put the submit button so submit or how about upload just to keep their names consistent so we have the upload here and then when they click on this button it will upload the form so let me go ahead and start the flask app make sure that's working and let me just put it into development mode so the server restarts and we see better errors and i'll go over to the browser and run this okay so we see it's not found because it's looking at the index so i have to go to slash upload and now we see our form here so let's go back to the code and i need to go to app.pi and what i want to do is open up app.pi and i want to write the code to handle uh processing the file so first i need to get the file and then i need to send it to my process that i have so first i'm going to bring in some code from the flash documentation so i'm just going to copy it over it just makes it easier to handle files like this and i can just put this i can just put this at the top so outside of create app is fine and all this is doing is it's saying that the only types of files that are going to be allowed for this particular thing are csv files so i have a set here i could have defined it with the curly brackets but i just have a list and then converting it to a set and here this function allow file will just make sure that the file that's uploaded the file name contains dot csv at the end and then also i want to import a utility from workserg and i just noticed that my url for disappeared up there so i want to import from workserver which is the library that flask is built on top of and it has this function called secure file name so from workzerg.utils i want to import the function secure file name and then next what i can do is inside of this if block for posts i want to get the file so to get the file i use requests.files so not request.form and i can just put this into a variable let's just call it file i know that it's kind of like a special word but in this case it's going to work so file is going to be request.files file so this gives me the actual file object that i can use and i can check to see if this exists so if file and i can also see if it is allowed so allowed file and i can pass the file name so file dot file name will give me the name of the file and then i want to convert the file name to a secure file name so secure file name like what it does is imagine someone has a file called mycsv or let's say like um home my cs or mydata.csv right so you see how there's a slash like this is not a good format for a file name so what secure file name would do is it just removes the slashes so if you were to use this somewhere you wouldn't accidentally end up touching something on your system like you're just dealing with the file itself not any path so secure file name will remove that so what i can do is i can say file name is equal to secure file name and then file dot file name so for our demo it's going to return the same file name but if there were slashes in there it would remove the slashes and then next what i want to do once i have this file name this new file name i want to create like a new file name based off this that has the date right so i need to import date time so we can go up here let's just put it here so from date time import date time and what i'm going to do with this is i'm going to basically put all the files that were uploaded into their own directory so i'll create a directory called input here and it's just going to save all the files that get uploaded to the system so i can create a variable called like new file name and then what i want to do is i want to convert this old file name to a new file name so the first thing i need to do is take the old one so the old one is file name and i want to split it on the dots so i want everything before the dot so i'll do square bracket zero so that gives me everything before the dot when i split it then i want to add an underscore and then i want to add the date time i could make this an f string let me make this an f string so this will be f and then this part goes inside right and then this is going to be followed by an underscore so the the name of the file underscore and then i want the the date time so i'll convert the date time to a string so all i need is date time and then dot csv at the end right so it's going to take the original file name before the csv part have an add an underscore put the put a timestamp and then dot csv and then finally i want to save this so to save i can take the original file object so file and then call dot save on it and then i need to specify where i'm going to save this file so i'm going to do os so let me import os import os so os is just my operating system library so with os i can do things like os.path join and then i want to join the input directory with this new file name so it's going to save them to the new input directory with the new file name file.save is taking the the actual contents of the file and it's changing the name here by saving it to this particular location okay so that saves the input so let's try this let's try to return just uh upload it and we'll see what happens so right now my input directory is empty if i go over here and i see i have an error with my f string so let me just go and fix that really quick we see there's an unmatched paren and it's because of the types of quotes that i use here this should be double quotes instead of single quotes because i use single quotes on the outside so let's go ahead and try that again and now it works so i'll just pick a file so product data same file i'll hit upload tells me upload it i'll go back to vs code and now we see in my input i have a new file called product data so if all you needed to do was save files as they're uploaded like this is the way you can do it in my case what i want to do is i want to process the file so what i'm going to do is i'm going to import the function from my script so let's just open that so this is just a function process csv i'll go over to app and i'll import it so from scripts import process csv and then what i want to do is i want to pass the new location to the process so let me make this a variable let's say save location and then we'll copy this and move it here or cut it and then we'll put it inside of file.save here so this is where it's going to be saved and then what i want to do is with that process file i'll pass the save location to it and i'll create a variable called output file so this is uh the name of the file in the location where it's saved and one thing i can do is i can allow the user to immediately download this file i can say return and then send from directory and the directory i want to send from is output and the file that i want to send is the output file right and then if i go up here i can import send from directory so let's take a look at what that looks like i'll upload the file again hit upload and now we see it immediately downloads the file so it processes the file and it returns like the the updated product types file to me so i can see like the sums of all the prices across the product types so if this were a long process you probably wouldn't want the user to download immediately so what we'll do is we'll redirect to this download endpoint instead and i can remove this uploaded and i can just go ahead and create this so i'll create a new routes so i need to make sure it's still inside of the create app function so app route and then download and what i'm going to have for download is like a list of all the files so what i can do here is i can say render template and i should return render template so return render template and then i want to return a file called download.html which i'll create in a moment and i'm going to return like a list of all the files that i have in the directory so files os lists directory and then output and i'll go ahead and create a new template called download.html and in here all i'm going to do is loop over those files so for file name and files in for what i want to do is i want to create a link to the file name so i'll leave that part blank because i need a place for the link to go to which i don't have just yet and i'll put the file name here so let's see what this looks like if i go to download we see here i just have one a product type because i've only done the process once if i do this again and upload now i have two right and they're on the same line so i'll put a line break just as a way to visually separate them and we see i have both files here so the goal is i want to click on one of these and it downloads the file for me so to do this i can create one more route which will be called well the endpoint will start with download as well but it's going to have a file name here so file name and then we'll call this download file and then it's going to take in the file name as a parameter and all i want to do is just call that syn from directory again so send from directory the directory is output and then the file name is going to be the file name just like that and then inside of here i just need to add in the href the url for so url for download file and then the second argument is the file name so file name equals file name here close that out and then put the curly brackets again and now i can go here click on one of the files and it downloads to my system and i can download either one so we see i can download both files so that's basically it that's all i wanted to show you for this video um i know it's an example like typically your process wouldn't be that simple and then probably wouldn't want to just have a screen where people can download files from the file system instead you might put them in a database and then inside of the database you'd have a reference to the location of the csv file it could be on your file system or it could be on a service like s3 so that's what i want to show you if you have any questions about anything that i've done here feel free to ask i'm going to put the code in the description below so you can go see the code if you want to take anything here and use it for your purposes so that's it for this video if you like this video please give me a thumbs up and if you haven't subscribed to my channel already please subscribe so thank you for watching and i will talk to you next time

Channel: Pretty Printed

Views: 23,997

Keywords: flask csv, flask, csv, python, tutorial

Id: BP8ulGbu1fc

Length: 17min 26sec (1046 seconds)

Published: Tue Jul 05 2022