How to Create and Use Virtual Environments in Python With Poetry

if you've been writing python code for a while you probably had to deal with resolving conflicts between different packages and versions that's what we call dependency hell most ecosystems actually have this problem you have all sorts of libraries and packages that everybody uses but there's also different versions of the programming languages operating systems and so on to solve this you need a separate and isolated environment that you have full control over that avoids conflicts with globally installed packages or python interpreters and that's easily reproducible in variety of context that's what we call a virtual environment it's a sort of metaverse if you will but then useful and you don't need to wear two ridiculous monitors in front of your eyes and you get to keep your data come to think of it it's not really metaverse at all so scrap that today I'll show you what I think is the easiest way to do this using a tool called poetry before you start a new project it's important to think about how to set it up and how to organize things in terms of design I have a free guide that helps you with this you can get it at Ariana called slash design guide contains the seven steps that I take whenever I design a new piece of software and I hope it helps you avoid some of the mistakes that I made in the past so iron.codes slash design guide the link is also in the description now let's Dive In before I talk about virtual environments I'll first show you the code example that I use today actually the code itself is not really that important because we're not really going to do anything with it it's just to have some sort of placeholder project that I can create a virtual environment for so in this case what I'm going to use is a file called ID generator that has a couple of functions to generate a variety of IDs and next to this I have a couple of tests for these functions that's in a separate test folder that uses simply the unit test and buy test packages so again I won't dive deeply into the code here because it's not that relevant I'm purely using it to set up the virtual environment in a few minutes so what is exactly a virtual environment well it's a folder structure that contains an isolated environment of python and its dependencies so when you create a virtual environment then that virtual environment is going to contain a copy or Sim link to the python binary it's also going to contain a pi VM configuration file that contains information about what kind of virtual environment this is it also contains a site packages directory that contains the extra dependencies that you are going to need for your project so it's really a lightweight isolated environment that you can just quickly set up and throw away again when you no longer need it the typical things that you will do is that you will create the virtual environment which creates the folder structure you can activate it so that you're now inside this virtual environment you can add dependencies or remove dependencies from that virtual environment and you can deactivate it again and of the tools that help you build these virtual environments more or less follow the same types of processes so if you want to start using virtual environments there are a bunch of different tools you can use the most basic one is vmf which is part of the Python standard Library since python 3.3 and this basically allows you to create a virtual environment it doesn't offer anything extra like dependency management so you would use this together with Pip so it's not ideal another two you could use this virtual amp which is a third-party tool it's actually one of the first tools available to allow allow you to create a virtual environment and part of the functionality of virtual amp is now actually also included in vmf another two is by end which Focus specifically on having different versions of python available on your system this is sometimes useful if for example your OS ships with an older version of python and you want to use a newer version but you don't want your OS to potentially break because it depends on things that were available only in the older version of python then you have a tool called pip amp which aims to bring together pip virtual amp and Bion and basically have one tool to do all of those things yet another tool you can use is condom which is an open source package dependency management system it specifically aimed at data science and scientific Computing and this also provides a way to manage dependencies in Virtual environments and even includes possibility to manage non-python packages however the tool that I'm going to use in this video is poetry I actually really like poetry for a few reasons the first one is that it's actually really simple it really simplifies all the steps of creating environments dealing with them adding dependencies removing them it's it's really simple to use I'll show you in a few minutes configuring it is really easy and it does a lot of things for you automatically which is a big improvement over just using pip and requirements.txt file for example actually I'm considering using poetry for all the code examples in my future videos so I think that will make it easier for you as well to use the code that I use as an example and have clear idea of the packages and the python version that you're going to need in order to run it another reason why I like poetry is that it not just deals with virtual environments and manages dependencies for you you can also use the publish your package to buy buy so it's really a all-in-one tool to deal with everything that you need and finally there's performance so poser is quite fast and efficient it uses for example a lock file so that it doesn't have to reinstall dependencies all the time so that's another reason for me to use it in my projects so how does poetry actually work well it's really really simple so I'm just going to switch to my console here so this is everything that's in the project so how do you actually use poetry well that's really simple you just have to install it as a first step and you do that by calling pip install both free if you write it correctly and of course I already did this so it says a bunch of times that the requirement has already been satisfied but this is how you basically install it and then getting set up with both 3 is actually also really simple so if you have a new project then what you need to add is a Tomo file a Pi Project dot Domo file that's what you see here there's an example of what this looks like so there are some settings about the name of the package the version description the license you see that there's also the dependencies in this project file there's some other requirements and you don't have to write this yourself you can actually generate it by simply calling poetry in it and then it's going to ask you a few questions and then we'll generate this Domo file for you but if I called here then obviously there is already a Tomo file so it doesn't do that but that's how you can set it up just in a minute so once you have this project description file what you can then do is tell poetry to create a virtual environment and install the necessary packages and that's also really simple we do poetry install and now it's going to create the virtual environment you see that it's also installing the packages so we need a couple of things like Pi test and bsom and as you can see those are actually defined here as dependencies so now you've created the virtual environment and you can also get some information from poetry about where that virtual environment is and what it looks like by simply writing poetry and info and then it's going to print out information about the environment and if you just want a path you just write minus B and then you're only getting the path this is sometimes helpful so it basically shows you some information about which python version is being used and where the virtual environment is located so you see that this virtual environment is actually in some local cache folder somewhere and that's maybe not ideal what I typically like to do is that my virtual environment is part of the project folder and that's actually an easy way to change this in poetry so the first thing that I'm going to do is simply delete this virtual environment again so I'll just switch to my finder app and this is the folder where we have the virtual environment that both we created so I'm just going to delete this because I don't want to have it here so what you can do if you want to make sure that virtual environments are created inside your project folder is to tell poetry that you change the configuration setting virtualms dot in project and when I select this I can set this to true and now when I do both reinstall you'll see that it now creates if I scroll up here it has now created a DOT VN folder which is the virtual environment folder structure that I mentioned in the beginning and there you see we have things like binaries we have a library with site packages you can also see that here and this is basically where all the dependencies are being installed like bson for example so let me close this again and now we have the virtual environment so how do you actually run our programs inside the virtual environment well with poetry that's also really simple we simply write poetry shell and now we're going to open a shell within the virtual environment and now I can start running my code for example I can run the test by simply calling Pi test and now you see it actually runs the tests within the virtual environment now adding and removing dependencies is also really easy for example let's say I want to add the requests package so in that case I'll simply write post react requests and now it's installing the request package and what's nice is if you look at the Pi Project file you see that request package has now also been added here so it keeps the dependencies up to date for you automatically and of course removing it is just as simple simply write poetry remove requests and now it's going to remove that dependency again and it also removes it from the thermal file so this is a really easy way to keep track of the dependencies that are needed and they're all included in the Pi Project Tomo file so how do you get out of this virtual environment shell well just write exit and what you can also do is you can ask poetry which environments are active so I'm going to list now the environments you can have multiple environments if you like so there is now dot VM which is the activated environment let's go back into the shell again so now I'm back into my virtual environment shell instead of just exiting the shell you can also deactivate the virtual environment by simply writing deactivate and now you're out of the shell and the environment is also deactivated removing the virtual environment is actually really simple because everything is in a single folder so if I pick this folder and I do delete now the virtual environment is deleted and I can create a new one if I like like I said before poets will also supports building and Publishing your package on Pipeline which is really nice so in order to get started with this you first need to tell by where to publish your package too so again you use poetry config and then you can indicate the name of the repository for example test bye bye and that's https test dot bypi.org slash Legacy so now I've made this configuration what you then can do is go to the test by repository and get a token that allows you to publish packages to the report story and set it in poetry by simply writing poetry config and then you name the token let's say Pi Pi token DOT test bye bye and then you copy the token so that's typically bye bye Dash and then the rest of your token I don't have the token here so I'm not going to do it now but this is basically how you do it so once you've done this you can tell poetry to actually build the package for you and you simply type poetry builds and that's going to build ID generator package like you see here it also includes the version number and now publishing is also really simple you just write poetry publish and then you provide the repository which is test by pie I'm not going to do that now because I don't have token but this is basically how you do it and if you don't want to do the build step separately you can also write here a dash dash build and then it's also going to build the package before you actually publish it to bye bye there's a couple of things you need to think about when you're working with a tool like poetry one is that of course you have to make sure that you manage your dependencies properly so you have to make sure that the dependencies that you installed that are up to date that the version numbers are incremented whenever a new version is updated off the package so both accounts this installed package automatically to the project file the Tomo file but for example if you're using pip and then this doesn't happen automatically then you have to add it yourself to requirements.txt and of course when you're using virtual environments you need to make sure that when you create the virtual environment that you actually install the dependencies properly so if you forget that you're going to get module mode found errors another thing you shouldn't forget of course if you work with virtual environments is that you need to activate the virtual environment and enter the shell before you can actually run the code otherwise it's not going to work what I find typically works best for me is to have the virtual environment folder inside my project folder so make sure you change that poetry configuration setting when you install it another thing you need to be aware of is that sometimes there are compatibility issues leading to you not being able to use a package that easily for example there are packages like Pi Auto GUI that rely on some system level packages and these system level pack packages are not going to be available in the virtual environment unless you have already installed them also there are packages that use C extensions to increase performance like numpy for example so in that case if you use a virtual environment you probably have to make sure that things are compiled and those C extensions might actually link with other system installed packages so that might also result in some problems when you try to run the code so it's not completely isolated the final thing you need to watch out for is that virtual environments can potentially take up a lot of space so if you have lots and lots of different projects use lots and lots of different environments you have to make sure that you have enough disk space for example here you can see that the VN folder is only 10 megabytes but if you install lots of dependencies then that might get really large so if you're running out of disk space make sure that you delete virtual environment folders that you're not using anymore overall though I think virtual environments are really nice solution as they allow you to create a really nice isolated environment which makes it easier to share your code and I hope this video gave you a quick overview of how to use toolite poetry to make your life easier like I've briefly shown you poetry also allows you to build and publish packages on Wi-Fi another tool you can use for that is setup tool so if you want to learn more about how to build and distribute your package and share your package with other developers watch this video next thanks for watching and take care

Channel: ArjanCodes

Views: 117,629

Keywords: virtual environment python, virtual environments python vscode, virtual environments python, python virtual environments, python virtual environment, python virtual environment explained, virtual environment, python venv, virtual environments, python virtualenv, virtual environments explained, python virtual environment mac, python tutorial, python poetry, poetry python, poetry python 3.11, poetry python tutorial, poetry python package, poetry python mac, python packages

Id: 0f3moPe_bhk

Length: 14min 42sec (882 seconds)

Published: Fri Mar 24 2023