Hey guys,
In this video, we will build the following web application from scratch using Python. In particular, we will be using the open-source
framework streamlit. The beauty of streamlit is that you can create
entire web applications without knowing HTML, CSS or Javascript. On this website, you can keep track of your
income & expenses. Just select a period, and enter your numbers
in the different income and expenses categories. Additionally, you could also insert a comment
down here. After clicking the save data button, your
inputs will be saved to a NoSQL database. If you now navigate to the data visualization
tab, you can retrieve a period from your database and plot it in an interactive Sankey chart. In this chart, you can easily see how you
spent your money. While coding out this solution, I will show
you some very cool tipps & tricks in streamlit, like this navigation menu and how to create
your custom colour theme for your app. Additionally, you will learn how to hook up
your streamlit app with a NoSQL database and, of course, how to create an interactive Sankey
chart. Even if you do not plan to create an income
and expense app, I am sure you will get a ton of value from this video which you can
apply for your next streamlit project. OK, and without further ado, let us get started. First things first, we need to install the
required packages. Therefore, open up your terminal or command
prompt, type 'pip install streamlit and plotly' and hit enter. We will use plotly to create the Sankey chart. With that in place, let me open up a blank
Python file. As the first step, I will import streamlit
as 'st' and 'plotly graph objects' as go. After that, I will create a settings area
at the top of the script. Here I will define some basic settings. As shown in the intro, I want to keep track
of my income and expenses in this web app. Therefore, I will create two lists for the
different income sources and expenses. Later in the video, we will use a for loop
to iterate over the lists to generate the actual input fields. Next, I will define the currency and page
title. For the extra pop, I like to use an emoji
for the page icon. Just head over to the following website. I will also leave the link to this website
in the script. Pick any emoji you like and use this shortcode
in your streamlit app. In my case, I have picked the following emoji. After that, I will add a setting for the layout. By default, the streamlit app will be centred. However, you could also change this value
to wide if you want. With those variables in place, let us set
up the basic page configurations. In particular, I will use our settings for
the page title, page_icon and layout. After the page configurations, I will insert
a title onto the page. For this example, I will simply take our page
title, add a space and concatenate it with our page icon. OK, before going further, let us see what
our web app looks like. Therefore go back to your terminal. Ensure that you are in the same directory
as your python file. In my case, my python script is located on
the desktop in a folder called 'income_expense_tracker'. To start the app, type run streamlit, followed
by the name of your python file. After pressing enter, streamlit should open
up your default browser and display your app. Alright, here it is—our website with the
page title, icon and title. By the way, streamlit will pick your default
system settings. As I am using dark mode on my machine, I have
a black background and white text. So, do not worry if you have a white background
with black font. We will customize the theme anyway later in
the video. OK, and with that said, let us carry on building
the web app. In the next section, I will define the drop-down
values for selecting the period. So, I want a drop-down list with the years
and months. I could now go ahead and write all months
in a list. However, we could make our lives easier by
using the built-in calendar & and DateTime modules. Both modules are already included in your
Python installation. So, instead of hardcoding the years, I can
get the current year by using 'DateTime.today' and extracting the year. If you want, you could also add the next year
by adding one. Or use the previous year by subtracting one. To create the list of months, you can use
'month_name' on the calendar object. The first item in the list will be empty,
so January starts from index position 1. OK, now we have everything we need to create
the actual input form. Above our input form, I want to inform the
user that the data entry should be done in the currency, which we have defined in our
settings. Then I will create the form by typing 'st.form'
followed by a key. Each form must have its own key. Yet, this key is not displayed to the user
in the interface. Optionally, you could also add the option
to clear all input fields after submitting the form. Within our form, I will set up two columns. In the first column, I will insert a select
box to select the months. Later, when submitting the form, we want to
retrieve the month the user has selected. We can do this with the help of the 'key'
argument. The second column will be very similar, but
this time I want to display a select box to pick a year from our list. As our web app is still running, I can now
simply save the script and refresh the browser to see our updates. OK, and here we have our header with the drop
lists for the months and years. Additionally, streamlit shows us a warning,
as we have not implemented a submit button yet. But we will take care of that later in the
video. For now, I just wanted to check if the select
boxes are working. To visually separate the period selection
from the next input fields, I will add a divider using three dashes. Next, I will group our income categories in
an expander element. Now, I will show you a nifty trick to create
the different input fields for the income categories. You just need to iterate over each item in
our income list. Within each iteration, I will make a streamlit
number input. For the label, I am using an f string. Additionally, I can also set the minimum value
and format. In my case, streamlit should convert the input
to integers. By default, the number input widget has two
buttons to increase or decrease the number. For this feature, you can also set the step
size. And last but not least, you also want to define
a key. For the key, you need to use a unique value. In our case, we could just simply pick the
income name. And that is all there is to it. Let me quickly do the same for the expenses. So for each expense in expenses, I will create
a number input widget. And with that in place, let me save the script
again and refresh the page. Now we have our two expander elements. If I open the income bracket, we can see the
input fields for the different categories. Likewise, we will have the same for the expenses. As mentioned, we could set the values by using
the following buttons. Now that we have this in place, I will add
an expander for the comments. To input a comment, I will use the streamlit
text area widget and leave the label empty. Optionally, you could also set a placeholder. After our comment section, let us take care
of the submit button. First, I visually separate it by using a divider
again. Then I will assign the 'st.form_submit_button'
to a variable called 'submitted'. When this button is clicked, it will return
True. So, we can check if the button was clicked
by typing 'if submitted'. In that case, we want to send all values from
our form to a database. But before doing that, we need to get the
values from each input field. We can do this by accessing the key of each
widget, which is stored in the Session state. So, as an example, I want to save the period
in the following format. Like a dictionary, we can get the values from
the session state by using the key. In my case, I will concatenate the year and
month with an underscore. I will follow a slightly different approach
to get the values from our income section. Instead of typing out all income categories,
I will use a dictionary comprehension. So, as you might remember, we used the income
category name for the key. This means I can simply loop over our income
list and get the value for each item by using the session state. And to map the value to the respective category,
I am using a dictionary. I hope it is not too confusing for you. I will also print out the income & expense
dictionaries to make it clearer. But before doing that, let me remind myself
that we want to insert those values later into a database instead of printing them to
our app. Also, to let the user know that the data has
been saved, I will include a success message. Now that we have this in place let me refresh
our website. To test it out, I will insert random values
for the income and expenses and provide a short comment. If I click the Save Data button, we can see
our income and expenses in a dictionary. As the next step, let us plot the data using
a Sankey chart. As before, I will set up a header and another
form. Later in the app, I want a selection box that
only displays the periods which we have saved to our database. As I haven't set up the database yet, I will
insert another 'to do'. However, to continue with our script, I will
just hardcode the period for now. So, in the selection box, I will simply type
'March 2022'. As I said, we will fix this hardcoded period
later. After selecting the period, I want to have
another submit button with the label 'Plot Period'. If that button gets clicked, I want to get
the data for the selected period from our database. Once again, as we do not have our database
yet, I will simply type out an example. So, we will have a comment and two dictionaries
– the income and expenses. Before plotting the numbers, I will create
simple metrics like the total income by summing up the income values. Likewise, I will do the same for the expenses. The remaining budget will be simple the total
income minus the total expenses. I want to display those numbers in three separate
columns. In column 1, I will use the widget 'metric',
followed by an f-string, where I am concatenating the total income with our currency. I will do the same in the other two columns. So in the second column, I will write the
total expenses, and after that, I will display the remaining budget and the comment for the
respective month. And before we continue, let us see how it
looks. After a quick refresh, we can now select a
period. So far, we have only our hard-coded value,
so March 2022. If I now plot this period, we can see our
metrics. Let me maximize the window so we can see our
three columns. OK, cool. Now that we have that, let us create the Sankey
chart. To explain the creation of the Sankey chart
better, I have created a Jupyter Notebook. The only purpose of this notebook is to explain
how we can create the chart. Afterwards, we will switch back to our Python
file. To get our feet wet, let us start with a simple
example. After importing plotly as go, I will execute
the following cell. The hardest part to understand are the following
lists. Let me explain. The label is quite straightforward. We have got here two different incomes and
expenses. In between, I have added an extra item called
'Total Income'. Next, we have the source list. In this list, we are defining the starting
point for each item in the list. So, 'income' is in index position zero. And the source will also be from index position
"0", so the same index. Income should flow to 'Total income'. As total 'Total income' is as index position
2, the target will be two. And for the value, I have picked 10 USD. Next, we will have 'Other Income' at index
position 1. So, the source will be one, and the target
will also be 'Total Income', so 2. For the value, I have also picked 2. I hope I have not confused you too much, but
stick with me. We also need to map 'Rent' and 'Food'. Unlike the 'Income' & 'Other Income', they
should start from 'Total Income', so the source will be two in both cases. And the target should be their own index position. So, three and four. For the values, I have picked 6 for rent and
4 for Food. OK, this was the harder part. Once we have our lists, we will create and
convert them into a dictionary. Then, we can also apply some styling within
the node, like the padding and thickness. And after that, we can create the Sankey chart
of the links and nodes. And finally, we can plot it. OK, so this was the simple example. Let us now move to our specific use case. As you remember, we have got an income and
expense dictionary. The challenge is now to create the lists dynamically
with those dictionaries. So, for the income and expenses above, the
lists should look like this, without hard coding any numbers. Let us start with the label and value list. For the label, I will simply list all keys
from the income and expenses and combine it with 'Total Income' as seen before. The values are also straightforward. Instead of the keys, I will get the values. The source and target, however, will be a
bit more tricky. The first part of the source list will be
our income. As before, the source should be the respective
index position for each element. And in the second part of the source list,
the expenses should start from the total income. For our example, the source list will look
like this. Now that we have got this, I will do something
very similar. All the incomes should have the Total income
as the target. The expenses, on the other hand, will have
their own index position as their target. I know this might seem confusing, and I have
not explained the different list comprehensions in depth. However, I will also upload this Jupyter Notebook
to Github so you can play around with the figures. Also, I am sure there might be better and
easier ways to create those lists. Let me know in the comments if you have a
better idea. And with that said, this is how our final
script looks. We can now create the Sankey chart for as
many income and expense categories as we want. Just a quick note: I have also changed the
colour here. And with that, let me switch back to my Python
file and write down all the lines to create the chart. The margin is the only thing I have changed
compared to the jupyter notebook. By default, every plotly chart has a certain
number of margins, which we can overwrite in the update layout method. To display the chart, I am using "st.plotly_chart"
followed by our figure, and I want to use the entire container width. Alright, let's give our website a refresh
to check out our chart. If I click the plot period button, we will
have our metrics and the Sankey chart. Let me maximize the window so that we can
see the chart better. OK, and here you have it. A nice interactive chart where we have a clear
overview of our income and expenses. So far, we have put everything on one page,
so the input fields and visualization. For the extra touch, I would like to add a
navigation bar at the top of our app. For this, we will need another package. So back in my terminal, I will type 'pip install
streamlit-option-menu' and hit enter. Once you have installed this package, go back
to your python file and import 'option menu' from 'streamlit option menu'. After importing this package, I will create
the navigation menu below our title. By the way, I already have a separate video
on how to use this package. In that video, I will explain the different
options more deeply. With that said, let me initialize the option_menu. For this example, I do not want to have a
menu title. For the navigation options, I will choose
'Data Entry' and 'Data Visualization'. For each option, I can also insert an icon. For the data entry, I will use a pencil, and
for the data visualization, I will pick a chart icon. To check out all available icons, head over
to the icons section on bootstrap. Here you can search for an icon. Once you find one you like, click on it and
use this name in your streamlit option menu. OK, back in the script, I will set the orientation
to horizontal. And with that, let me save the file and refresh
the web app. Great, now we have got our navigation menu. So far, those buttons do not do anything. We just store the user selection in a variable
called 'selected'. To only display the data entry form, I will
add an if condition, which checks if we have selected "Data Entry". Afterwards, I just need to indent the following
lines by one level. Next, let us do the same for the Data Visualization. Just insert the if condition and indent the
code afterwards. OK, now it should work. After refreshing the page, we can only see
the data entry form. If we click on Data Visualization, we switch
to our plotting section. OK, now that we have this also in place, let
us customize the styling of our app. I will first hide some default streamlit elements,
like the menu icon, the colourful header and the footer note. To do this, I will write some CSS code to
set the visibility of those elements to 'hidden'. To inject this CSS code into our app, I use
a markdown widget and set 'unsafe_allow_html' to True. So, if I refresh the page now, you will notice
that the colourful header and the menu icon have disappeared. Next, let us create a custom colour theme
for our app. Therefore, go back to the root directory of
your app. Here, you want to create a new folder called
'.streamlit'. Inside this folder, create a file with the
name 'config.toml'. Open this file with your preferred text editor
and create a new section called 'theme'. Under this section, you can set the different
colours. Let me shortly type out all available options
here. I think it might be easier if you grab this
code from my GitHub repo instead of typing it out yourself. For each option, you can also see what it
will change in the comments. So, feel free to adjust those colours to your
needs and liking. One important hint: Nothing will change if
you save this file and refresh your app. You actually need to rerun your app to apply
the custom theme. So, back in my terminal, I will stop the app
by pressing CTRL + C on my keyboard, and then I can rerun it. After your app has been loaded, you can see
the changes. OK, this is looking great so far. The only thing left to do is to set up the
database to store and retrieve our input values. There are many great databases available,
but I like the NoSQL database from deta. Just head over to their website at "deta.sh". The database from deta is super easy to set
up, and it is free. To create an account, click on 'Join Deta'
and follow their sign-up procedure. Once logged in, you will see a page which
looks like this. Deta offers a couple of different services,
like a cloud environment for your app, a database and file storage. For our app, we are only interested in the
database. Therefore, click on 'Base' and then 'Python'. Here you can find a quick guide on how to
get started. The first thing you want to do is to install
the deta SDK for Python. Therefore, open up your terminal or command
prompt type 'pip install deta' and press enter. Once installed, go back to your dashboard,
and click on 'Settings' and 'Create Key'. As mentioned here, this project key will only
be shown once, and you should store it in a safe place because anybody with this key
could access your database. With that said, let me copy this key to my
clipboard and go back to the root directory of my app. To keep our code organized, I will create
a dedicated Python file which will interact with the database. I will call this file 'database.py'. In this file, import deta from deta. Then, I will store the key we just have copied
in a variable. This is not really a secure place. Everybody could see your project key if you
uploaded this file to GitHub. We will take care of this issue in just a
moment. But for now, let us move on and initialize
a deta object with our project key. We can now use this object to create or connect
to a database. Simply type 'deta.Base' followed by the name
of your database. I will call mine 'monthly reports'. And with that, we are ready to use our database. There are just three things I want to do:
Insert the values for a specific period into our database, fetch all periods to fill our
selection box, and lastly, I want to get all values from a particular period to plot the
data. So let me quickly explain what I type here. To insert a period, I have defined four parameters. The period name, the incomes, expenses and
the comment. To insert those values, simply type 'put'
on the database instance variable and provide the values in a dictionary. The critical part here is the key. For the key, I have picked the period. So, this will be our unique identifier. In just a moment, I will show you an example
of this. To fetch all periods, you can type 'DB.fetch'. This will return an instance of a FetchResponse
class with the following properties. In our case, we want to get back all items. And finally, we can also retrieve only a specific
period using 'get'. To test and explain the functions, I have
created the following Jupyter notebook. As before, this notebook is just for testing
our code. For the actual app, we do not need this notebook. So, after running the first cells, let us
go through an example. We want to store four things when submitting
data in the web app. The period, comment, income and expenses. And the data looks like this. So, later in the app, we can use our 'insert
period' function. Let me run this cell and go back to my deta
dashboard to test it. If you refresh the page, you should see your
database on the left side. OK, and here it is – our first entry. If you click on the expenses, you can see
that deta saved the data in a JSON object. The neat thing is that you can manipulate
any entries directly from your dashboard. For example, let me change the blog value
to 500. Afterwards, I can save the changes. If I now go back to my notebook and fetch
all periods, I can see the updated value for the Blog. Instead of fetching all periods, you can also
get back a single period. OK, now that you have seen how this works,
let me go back to the deta dashboard and delete our example entry. We could also validate this by trying to get
back the April data one more time. This time, we will back an empty response. This is all working fine, but there is one
thing I want to change. If we go back to your 'database.py' file,
we have still the hard-coded project key here. Let us now move this key into an environment
variable. For this, I will use a small helper package,
which you can install by running 'pip install python-dotenv'. Once you have installed this package, go back
to your script and import os, a core python module, and from dotenv, import load_dotenv. After those imports, we can load the environment
variables by typing load_dotenv, followed by the file path. In just a moment, we will create this '.env'
file. Later, we will need our key in that '.env'
file. So, let me copy it to the clipboard. Instead of our hard-coded key, I will now
write 'os.getenv' and then 'Deta_Key'. OK, now let us create this env file in the
root directory of the app. You can simply create a text file and rename
it to '.env'. In this file, type 'Deta_key' followed by
the project key and save it. Now that we have our .env file, you want to
avoid uploading it to GitHub or any other platform. Therefore, also ensure to create a gitignore
file. I usually use the service from gitignore.io. Just head over to their website, type python
and click on create. Then copy the whole text, go back to your
project folder, create a new text file and name it ".gitignore". In this text file, paste the text you just
copied. If you scroll up a bit, you will see that
git will ignore any environment files. OK, now onto the last step. Let us now connect our streamlit app with
our database. Therefore, I will import our local database
file as DB. Afterwards, let me navigate to our 'to-dos'. We can now use our 'insert period' function
to insert our values, with the period, incomes, expenses and the comment. With that in place, I can delete the following
two lines. As the next step, let us get all the periods
from the database. For this, I will create a small section at
the top of the script. To get all periods, let me fetch all items
from our database. I am only interested in the key, so the period
name. Therefore I am using a list comprehension. Frankly speaking, I could have implemented
this function also directly in my database.py file. But anyways, from this function, I will now
return a list of all periods. So, if I scroll down a bit, I can replace
our hard-coded period name with our function and remove the 'to_do' marker. Once the user selects a period, we want to
get the data from our database. Previously, I have inserted here some dummy
data. So, let me delete the following lines and
instead, I will retrieve the data using the selected period. This will return a Python dictionary. From this dictionary, I will now retrieve
the different values for the comment, expenses and income. OK, and that is all there is to it. Let me save the script and refresh our app. To test it out, let me input some random data
into our entry form. Additionally, I will write a short comment,
and then I can press the save data button. If I now switch over to the data visualization
tab, we should see the entry for January 2022. And indeed, in the drop-down menu, we have
our entry. So far, we only have the January data available. If I select it and hit the 'Plot Period' button,
we have our metrics and the Sankey chart. So, let me add more data to our database. Back in the entry form, I will insert some
values for March 2022. Once done, I will press the save data button
again. As a result, we will now have two entries
in our selection box, which we can plot. Also, if we go back to our deta dashboard,
we will see our entries. Ok, now that we know that everything is working
you might be asking yourself how to deploy this web application to the internet without
exposing our database key to the public. And this depends on the hosting service you
are choosing. For example, if you are using Heroku, you
can configure the environment variables via the command-line interface tool or the Heroku
dashboard. If you want to deploy it to the streamlit
cloud, I will leave the link to the following blog article in the description below. In this article, you can find a step-by-step
guide on how to store and retrieve the environment variables using streamlit the cloud. OK, guys, and that is all for this tutorial. If you have any questions, let me know in
the comment. Thanks for watching, and see you in the next
video.
