Automated reports using python and jinja HTML templates

hi friends in this video we're going to talk about how to generate automated reports using Python and ginger HTML templates so using Ginger templates we can create HTML reports which are highly interactive and rich in styling this is an example report generated using Jinja and python you can see we've generated HTML table with some data in it and we have generated some list items and we have even generated interactive plot because it's HTML right in HTML you can have interactivity using JavaScript and CSS so since we are generating HTML reports these are rich in styling and interactivity all right let's get to know what is a ginger template one easy explanation of a ginger template is basically it is a string which has placeholder variables where you can inject data into that variables for example I've got a small Ginger template which is actually a birthday invitation so here you have placeholders with variable names and using python you can actually inject data into these variables so this is a ginger template and using python I can inject data into this Ginger template and I can create output something like this so I can substitute values into these Ginger variables and I can create outputs doing something like this so this is the ginger Syntax for substituting variable names you have to write double curly brackets and write the variable name here not only substituting data into variables in ginger templates you can even do for Loops let's see an example here suppose I got a python list top items so this is a python variable which has list of strings I need to render this list of strings in a ginger template so let's say I have a ginger template something like this so I can declare a fur Loop in a ginger template so here I am writing for item in top items since the variable name is top items I am iterating through this top items just like a python and for each item I am doing something here I'm just rendering the item so the output would be something like this top performing atoms item two item Z Item seven so this way you can even use for Loops in ginger templates all right now let's apply the same logic to HTML Ginger template that means the ginger template is basically just a HTML script with ginger variables and control statements so here we have replaced with the same example but inside HTML tags so I am declaring the top performing items is a h2 tag that means a heading tag and I am giving a unordered list and inside that around the list I am rendering each item as a list item so the output would be something like this top performing items and you will get an unordered list with list items so it's basically a HTML so the output would be something like this so we are getting a styled output because we are using HTML tags all right we have seen how Ginger can be used now let's see how can we implement it in Python to do Ginger templates in Python you need to install the ginger module so I'm gonna write python minus M pip install Ginger tool so now we are installing the ginger to python module it's installed in my PC now all right in a python code let's try to implement Ginger templating first let's try to take a string which is a ginger template so now I want to substitute values into this Ginger template first you create a template object so first you write import Ginger 2 in your python code and create an environment ginger2.environment and since you are using Ginger from a python string you can write loader equal to Ginger 2. base loader and from string the template string which you have declared just now so from the template string you have created a ginger template object and now just render a dictionary with the variables in This Ginger template object for example I have created a variable called context and this additionally which has a variable names name age and City and now I'm just rendering this dictionary in the template using template.render the context and then the output would be the rendered string output it's that simple to use ginger in Python let's try to implement this now I'm going to take a blank folder and I'm going to open it with vs code let's create a new python file I'll just name it index.pi first let's try to import jinja2 so I'm going to write import Ginger 2 let's try to declare our Ginger template string and now I am creating a ginger template object using ginger2 dot environment and the loader would be a ginger to based order because my template is just a python string and I'm saying from string this template string which you have created now so the data to be supplied in the placeholder variables can be provided as a dictionary for example I have created this variable called context and now the output would be something like render text equal to template dot render of the context let's try to see what the output would be I'm going to print that output so I'm gonna write print render text now let's try to run this example and here we got the output the variables of the context are being substituted in our output string so it's that simple to use ginger all right now instead of deriving the template from a string let's try to derive the template from an input file let's call it the template file let's get a new folder here and I'll name it templates and let's say to create a template here so I'm going to write it bio template Dot txt and let's try to copy this text in the template file so now I got my template file ready now let's try to remove this template string and now let's try to load the template from a file so this is how you would load the template from a text file actually so this time I'm gonna write ginger2.environment and we are going to use the loader as a file system loader because we are going to load the template from our folder and the folder would be slash templates because the template is present in the templates folder so I'm going to write dot slash templates or you can use the absolute URL of the folder and I'm telling get template biotemplate.html and here we have used one more thing called Auto escape so I'm telling Auto escape equal to ginger2.select Auto escape so what this does is it enables HTML escaping so in the variables being substituted you can give only strings or numbers you can't give HTML so you can't inject HTML using variables if you set this Auto escape equal to select auto Escape so this is good for security purposes of course in our example it's not required you can even remove this so that's it I am loading my template biotemplate.txt I am providing a context to substitute the variables and I am rendering it now let's try to run this file and you got the output you can see the output is actually text now you can use this text and create a text file out of it let's try to do that now so I'm just taking the output which is a rendered text and creating a text file called bio.txt and just writing that stuff into that file so let's try to run this now and here in the bio.txt I got the output so effectively what we did was we use a template which has variable placeholders in it and create an output which has the data in it so this is the basis of today's discussion we're gonna use some report templates where you have the variable name placeholders and for Loops or if statements and you will generate reports out of it all right let's try to see another fun example using Ginger so here in the templates folder I got a file called invite template.txt so it's just a simple message to invite some guests to an event so we have some placeholders here the recipient name the event date stream venue string and the sender name now I can use a python script to actually generate invites for multiple people from this single template so let's see how we can do that first you're gonna import Ginger 2 and load the template from the file system so I'm gonna write Ginger to environment file system loader in my template.txt and then I'm going to create a context object which has the variable data so this template has recipient name even date string venue string and the sendup name so we have the data in this context now I've kept the recipe name as blank because I'm going to use a list of recipients and render the template let's say these are the recipients so I have a list of guests names for each of which I am going to create an invite now let's write a for Loop and render the template for each guest so I'm going to write 4G in guests and I need to set the template context data for the recipient name so I'm gonna write template context of recipe name is the guest and then render the template and save the template so I am going to save the templates in a folder called invites I have created already the invites folder here so I'm gonna write invite slash the guestname.txt so I'm going to create a txt file for each guest name and write the invite output data there so that's how I'm going to create invite for each guest from a single template all right let's try to save this and run this and here you got invite for each guest so this way using python I've used a single Ginger template and I've created multiple outputs so the same strategy can be used to create reports you can have a single report template and based on that you can create periodic reports like daily reports weekly reports Etc all right till now we have used text based templates now let's try to use some HTML templates all right to demonstrate HTML reports I've got a simple example here we are going to generate a simple dummy sales report here so this is our template so in our templates folder I have created a HTML template file called sales report template so it's just a HTML file which has some style tags and inside the body we have something like a table and a list items and we got image let's try to study this template now we got a placeholder saying report for the report date string so this we need to supply in our python script and in the body we have an image for the logo actually so we will render images as base64 because that way you don't need to depend on external files so in HTML if you write something like this SRC data image PNG base64 and if you write the base64 image here the image data would be present in HTML itself you don't need to link to another image so this way your report would be a standalone and it will not be dependent on other files for images this way when you're shipping your HTML file you don't need to ship the image files because the image is embedded in the HTML so the image data would be populated in this logo image variable and again for the heading we are substituting the report date and in the table we are given the headings serial number item name cost per unit unit sold and revenue in the table body we will render each row from this list of objects called sales table rows and this should have attributes called serial number name cost per unit number of units and revenue so for each sales table object we will create a table row so that's why we are using ginger for Loop for item in sales stable rows and for each item which is an object we are rendering a table row here so this way we are populating data into a table from a list of objects in Python the same strategy is being used here we are creating a unordered list here and for each item in top items rows so there will be a list of objects called top items rows in our python script and we are injecting that into Ginger and for each top item Rose object which is item I am rendering a list object so each item would be rendered as a HTML list object here and then for the revenue bar chart the same strategy is used we are rendering an image and the base64 data of the image would be actually present in the variable sales bar chart image so this way I got a template for the report and the variables of the report are basically Ginger placeholders so when I inject data into this template I can generate a HTML report so we can use a scenario something like this fault generating daily reports weekly reports monthly reports Etc now let's try to create a python script where we can inject data into this HTML report so let's go to our python script index.pi I'm importing Ginger tool so first things first we got to load the template so I uploaded the template from the file system loader the templates folder and the name of the template is sales report template.html because the template is sales report template.html here and now I need to prepare some data right so in our template we have variable called report date string so I'm going to generate a string for today in this format so I will create a variable called today string so that I can use this variable for the report date string variable and then in our template we have something called sales table rows so I've created an empty list called sales table rows and I've generated some random data and I've ensured that there are attributes which are required to be rendered in the HTML so I need attributes like serial number name cost per units number of units Revenue so I've created random data with these attributes so now I got the sales table rules with the required objects that can be rendered in the HTML and then the next object is top items rows which is also a list of objects and each object is just a string actually so I am rendering the string here so from this sales table rows I am sorting them by revenue and getting the names of the top three so this code does that actually so in the top items I got a list of strings which are top three so till now we have generated the data which is a date string sales table rows and top items now let's talk about the images so we are going to load the logo image and we are also going to load the revenue bar chart image sales bar chart image so how are you going to do that I got the logo.png present in my templates folder so using this PNG image I need to create a base64 string out of it so I'm going to write with open template slash logo.png read bytes and then I'm gonna say base64 encode of reading this file object and then decoding it as utf-8 and giving that to a variable called logo image and now we are going to use this to render the image in the ginger variable so this is how you can actually load images in your Ginger templates as base64 strings so now to create the bar chart image I'm using the matplotlib python Library so matplotlip can be used to create charts in Python if you don't know about map.lib I've already made a video series on that and I will leave the link of that Series in the description ultimately what I'm doing here is I am creating a plot here and saving that plot which is the figure object as an image bytes and then I'm using that bytes to again create base64 encoded image string so now I have created the image string from a file and from a matplotlib figure so now I got all my variables set up so basically as a step 2 I have created the data the step one was loading the template the step two is creating the data and once my data is created I can create the context object so I will use the same Jinja template variable names here report date string sales table rules top item Source sales bar chart image lower image and assign the variables here now I got all the values ready in the context object the next step is really simple render the context in the template object so template dot render context and then you will get the text which is the report so now you need to just save the report so now we've got the report text which is a report content now just you need to save it so I'm saving it to a folder called reports sales report.html and then I'm just writing it to the sales report.html so that's it as step one we recruited a ginger template object from a template file in the Step 2 we have fetched all the data or created all the data for the report and then created a context object so that it can be injected into the ginger template and then the next step step three is to render the context in the ginger template and get the output text and the last step is just to save the text as the required report file so if you are doing something like daily reports you can just write sales report underscore the date something like sales report underscore the date string we already have the date sync today's stream so I'm gonna use that here today string so now the report would be sales report 9th in September and tomorrow it will be sales report 20th September and so on so you can generate daily reports very simply like this all right let's try to run our example now so in the reports folder we got sales report dot HTML so let's say to open the report so I'm going to open the report sales report.html and the report is basically a HTML file which has HTML content which is a table on ordered list and image you can see the image is loaded Standalone using base64 content of the image and the same way bar chart is also loaded here now let's try to see the output text actually so I'm going to right click here view page source and here you can see report for 19 September which is the substituted variable and the image source data PNG basis foreign this is all the basics for content of the image which is rendered in the ginger template variable and each row is being rendered using ginger for Loop here and the same way each list item is also rendered using ginger for Loop and the revenue bar chart image which is generated from matplotlib is also rendered as a base64 image string here so we have a standalone report file which is independent it doesn't even depend on external images because the images are also embedded in the image tag so now you can very easily distribute this HTML file to others so the people can open this in their browser and see the reports all right now let's take this report template a little bit further and let's try to inject bootstrap CSS into this so that we can demo the use of external libraries in our report templates so in the head I'm gonna inject the bootstrap CSS here so bootstrap is a very famous CSS styling framework so in the template I have linked to the bootstrap CSS and in the body I've used some CSS classes something like table tables type table border these are CSS classes of bootstrap so using external CSS libraries I'm actually adding styles to my HTML very easily so now we have modified a template to accommodate bootstrap CSS and bootstrap.js and I've given some standard bootstrap classes so that I can easily add additional styling to my templates so let's try to run our script again so I'm going to save this index.pi and run this again and the sales report is again generated let's try to reload this page and now you can see the page is styled using bootstrap CSS and with very less effort using external CSS Frameworks I can easily add styling to my reports now the report looks more styled using bootstrap but remember now your output template is not Standalone it depends on this URL for fetching the CSS and it depends on this URL for fetching the JavaScript file so if you want this report output to be truly Standalone and you don't want linking to other files so that you can easily distribute this file to places where there is also no internet how can I solve this problem so in our python script itself we can fetch the CSS and JavaScript files using the request module of python so I have written request.get the URL of the CSS file dot text and request.get the URL of the Javascript file dot text and now I have the whole JavaScript and CSS files content in my Ginger variable now let's try to render the CSS in JavaScript from Jinja instead of linking it to external files so CSS can be rendered using the style tag so I'm going to write style the bootstrap CSS variable which I have generated in the index.pi file let's try to comment out this CSS from external link now we are rendering the CSS from Jinja and the same way let's try to comment out the script tag and let's try to render the script from Ginger so here in the script track you can write the variable bootstrap.js so now the JavaScript content and the CSS content is actually being rendered from the python script so now your output report would be completely independent of internet and all the data would be present in the file and you can distribute this file independently to your stakeholders but when you run the script we are injecting CSS and JavaScript into your output report and if you remember in the ginger environment we are telling Auto escape equal to select auto Escape which actually enables Auto escaping for HTML so I'm going to write here Auto escape equal to false so I will allow injection of JavaScript and CSS into my HTML file so I'm going to save this now and then run this and now our report is generated and let's try to see the output report generated here I'm going to reload the output file and I'm not seeing any changes the output is generated now let's say to view the page Source here and here in the page Source I have rendered the whole CSS file as a style tag also I have rendered the whole Javascript file as the script tag and for this reason my file is 382 KB which is a bit more size if you use external linking the file would be very Less in size so be cautious while you use technique like this to render the whole JavaScript and CSS into your output files your output files might be bulky so use this technique of injecting the JavaScript and CSS into your templates only if you know that you don't have internet access if you have internet access you can just link the CSS and JavaScript from external files and your report would be lightweight all right we have covered how to use ginger templates for HTML reports how to use external libraries like bootstrap to easily style your reports now one more thing is pending which is interactivity since we are using HTML Pages we can easily add interactivity to our Pages let's try to see this bar chart this chat is basically just an image right now instead of rendering an image we can use plotly JS JavaScript library to actually render an interactive bar chart here let's try to do that now so in our sales report template let's try to insert a script for fetching plotly.js so plotly.js is a JavaScript library using which you can create interactive graphs in your HTML pages so I have imported the plotly library using the script tag in my template so the revenue bar chart is actually an image right so let's try to comment on this image and let's try to create a div called Revenue bar chart and then I am creating a script tag where I am actually initializing a data object and for each sales table row object I am inserting data into this data JavaScript object and then using plotly to create a plot in this div so this is basically plottilages so ultimately what we are doing is we have created a div and in the div we are actually using plotly JS to insert an interactive bar chart so this is basically JavaScript and the data for JavaScript is being supplied using Ginger templating so let's try to save this now and in our index dot file we don't need this sales bar chart image because we are actually rendering the image in the front end so you don't need to send this image now and we're also not injecting JavaScript and CSS here we have commented out those so we don't need to render these variables also so now let's try to save this and now let's try to run our script and now let's try to reload our output and here we got the revenue bar chart and this is an interactive bar chart generated by plotly.js so I am able to interact with this broad chart now I'm able to even zoom in and zoom out and all the standard plotly.js features are present in this report template and you can distribute this report template to your stakeholders and they can also interact with this report so this way using Ginger templates in HTML and python you can actually create HTML reports which have Rich styling and then we can also add interactivity to our reports using JavaScript so that's it guys this is how you can use ginger templates for generating HTML reports you know Ginger templates are used in web Frameworks like flask I've already created a playlist on flask web development in Python I will leave the link of that playlist in the description till now we have seen how to do variable substitution and for Loops in ginger templates you can even have template inheritance macros in ginger templates I have covered that in this playlist so if you want to know about how to do template inheritance template splitting and macros in ginger templates you can go to this playlist you can see I've created blog posts with all the instructions examples the source code for each scenario in this blog post so please be sure to check out the link of this blog post in the description of this video I also given the links to the official documentation so that you can do further reading please ask questions or post your valuable feedback in the comments section so that's it guys hope you like this video thank you for watching peace

Channel: Learning Software

Views: 3,688

Keywords: python, learning, beginners, tutorial, python_reports, reports_automation, jinja, reports, automation

Id: 32mE-TU7N1E

Length: 25min 19sec (1519 seconds)

Published: Fri Sep 22 2023