Essential AI Tools and Libraries: A Guide to Python, Git, C++ Compile Tools, FFmpeg, CUDA, PyTorch

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments
Captions
Greetings, everyone. Welcome to the Pillars of AI Application Libraries: How to Install Tutorial. If you are interested in generative AI, such as Stable Diffusion, such as large language models, web UI, such as voice cloning, such as text-to-speech, image-to-image, text-to-image, text-to-video, video-to-video, image-to-video, and anything that includes AI, this tutorial will help you tremendously. This tutorial is prepared for regular users, not for programmers. Therefore, if you are not a programmer, if you are not an expert, watching and following this tutorial will make your AI journey much more easier. I am going to show you how to install any Python version. I will install Python 3.10 and 11 and show you how to set your default Python version. I am going to show how to install Microsoft Visual Studio C++ Compile tools. This installation will fix the Microsoft Visual C++ 14 or greater is required error, such as the error that you are getting when you are installing Roop deep fake application, or any other application that compiles its own wheel on your computer. I am going to show how to install and setup FFmpeg on your Windows. So all of the applications that depend on FFmpeg will work without any issues, such as Whisper or such as video animation AI applications. This installation will fix all of the errors that are related to FFmpeg. I will show how to install CUDA libraries. In some cases, CUDA libraries are also necessary for compiling, for installing AI applications, AI libraries. I will show how to install Git and Git Large, which is necessary, mandatory to clone and install open source repositories, GitHub repositories. I will show how to do Git checkout, Git pull, Git clone, Git stash, Git stash pop. These will be also very useful in your AI journey. If you are interested in AI, you really need to know these commands, what they do. I will show also advanced stuff such as generate your own virtual environment, and I will show how to activate already installed virtual environment and install or uninstall any library because sometimes you may need to install specific libraries to fix the errors. And by watching this tutorial, you will learn that as well. I will show how to have multiple Python versions, how to set your default Python version, how to generate a virtual environment, or install any application with a certain Python version. I suggest you to have a single Python version, Python 3.10.11, but if it is necessary because sometimes it is necessary to have multiple Python versions, after watching this tutorial, you will not have any issues. I will show how to install any application and set its environment path variable so you will be able to utilize all of the applications. I will show as an example runpodctl, which is used to transfer files between RunPod and your computer. So, this file link will be in the description of the video and also in the comment section of the video. I will update this file if it be necessary. I am showing everything on a freshly installed Windows 10 virtual machine. So, this is a clean installation, and I will show everything on this machine so you will not have any issues to follow this tutorial. So, please watch this tutorial very carefully, entirely, and I am giving you a guarantee that you will save a huge amount of time that will be caused by the possible errors and mistakes that you will encounter in the future. And just for this tutorial. I installed a fresh computer. So, I will show everything on a fresh computer. None of the things previously installed and this will make it easier to follow this tutorial. So, let me first open a cmd. Cmd is command line interface command prompt. Usually, we work with cmd when we are working with open-source applications. When I type Python, you see there is no Python, and it is opening this page. But do not install from this page. Installing from here will cause you problems in the future. So, no Python installed. When I type git, you see git is not installed. Let me also show you my environment path. Now, environment path and variables are extremely important. Why? Because applications will look at environment path to determine where the application is installed. By application, what I mean I mean that, for example, the Python where it is installed or your CUDA libraries where it is installed, or FFmpeg where it is installed. So, you need to understand and learn how to edit system environment variables, and edit path. I just typed env into my search bar, and I am opening edit system environment variables. This is inside the control panel. Then click this environment variables button here, and you see this is a fresh installation so there aren't any set environment variables. Let me show you the path variable. This is where the applications by default look and there is also a system variables path. As you are seeing right now, it is also fully empty. You just need to add applications to the path here. I will show them in the tutorial. Okay, so let's first begin with installing the Python. Why Python 3.x you may wonder because this is the most widely supported and best working Python version right now. So, do not install Python 11, 3.11, or 3.12. You will have a lot of problems with the open-source AI applications. So, to download it, let's go to this link. Let me also make this full screen. Okay, so this is the Python 3.10.11 official version. Let's download Windows installer 64 bit. Okay, it is downloaded. Let's start it. Now. Follow here very carefully. You see there is add Python exe to PATH. You need to select it so it will add it to our environment variables path. Also, click customize installation, select everything here. Next, select everything here, and also change this path. Install it directly into your C drive like this without any space character. So, do not install like this. Do not have any space character. Do not have any special English character. So, I will make it like this. Python 3. Actually Python 3.10.11. Okay, let's click install. It will ask me this okay, let's click yes and just wait. Okay, so the installation has been completed. There is also this important question to ask. Disable path length limit. You should also click this. It will disable the path length limit. It is important. And now when I start a new command line, I type cmd into my search bar and click command prompt. Let me show you where I am typing it. So, you see I have a search bar here. When I type cmd here, it is appearing like this. So, I start a command interface, and now when I type Python, I see 3.10.11 Why? Because it is looking into the system environment path. So, when I open my environment variables, click environment variables, and it is not inside this path inside user variables because this is specific to that user, and we installed it for all of the users. So, it is inside this system variables path, and it is added to the very top. As you are seeing right now. See Python 3.10.11 scripts, and see Python 3.10.11. This is how the applications will find your Python. Okay, Python installation has been completed. Now, as a next step, let's install Git because Git is necessary to clone AI repositories, GitHub repositories, and install the stuff. So, click download for Windows. Select it like this: 64-bit Git for Windows setup. Start to download. Run the application from downloads. Click yes. Just click next. Next next, you don't need to change anything, just click everything next and install. Because the default Git installation is sufficient for us to install AI applications. Okay, now when I type cmd, and when I type git, you see git is installed. Okay, nice. Then, what should we install? We need to install Visual Studio C++ Compile tools and why this is extremely important. I don't know have you seen my Roop the deep fake tutorial. It requires InsightFace library. And InsightFace library requires C++ tools. Let me demonstrate to you. So, I do pip install InsightFace and let's see the error that we are going to get because we don't have the C++ tools installed yet. It is installing all of the necessary packages as you are seeing right now, and when it is compiling, we get this error. Let me show you that. So, the error we got is Microsoft Visual C++ 14 or greater is required and that is why installation of InsightFace failed. A lot of my YouTube followers have got this error, and they had a hard time to fix this. So, let's go to the Visual Studio C++ Compile tools. Open this link and in here, scroll to down. In here, you will see Visual Studio tools. From here, go to the build tools for Visual Studio. Click download. Open in the downloads folder. Right-click, run as administrator. Click yes. Click continue. Then you will get this screen. On this screen select this. You see Desktop Development with C++ and you don't need to select anything else. This is the latest version 17.8.3 and click install - install while downloading. So, this is a big package, but it will install everything that we need for us meanwhile downloading. It may take a while depending on your internet speed and your hard drive speed. Okay, the installation has been completed. Let's close this window. It didn't even tell us to restart the Windows. Now I want to test the installation of InsightFace one more time. Let's open another cmd and do pip install InsightFace and let's see we will get any error or not. We should not get any error. Also, you need to open a new cmd after the installation. New command line interface. It is using the cached pip files. Okay, it is building the wheel. This is where it compiles itself and it compiled I think. No errors. So, it used the C++ tools that we have installed. Okay, now it is running the setup.py file. You can also disable this cache of the tools. So, I added another link here that shows the pip caching. Sometimes, your cache may get corrupted so you may need to clear your cache. When you open the link that I shared here, you will see in the very bottom pip cache list, pip cache list setuptools, pip cache purge. So this will clear your cache. So the pip will redownload all of the cached files again. You can also disable cache with this command. So, let's see the installation. Yes, the InsightFace has been installed without any error. So, the C++ tools installation fixed the major problem that you might encounter. So, when you encounter this problem, Microsoft Visual C++ 14 or greater is required, this is the thing that you need to install. I also added a link for Git Large. So, ever if you need Git Large, download it from here. Install it by default. This will allow you to git clone large repositories like 5 gigabyte, 10 gigabyte files having repositories. Okay, it is already installed. Now FFmpeg. This is super important. So, when I open a cmd and type FFmpeg, you see FFmpeg is not recognized. So, how am I going to install it? Click this link. This is the latest version. You can also manually find this link. How? Open a new tab and type download FFmpeg. You will go to its official website which is here. When I click here. Click Windows, click Windows builds by BtbN. In here you will see the latest link. So this link is from there basically. The FFmpeg is downloaded so I will extract it into the downloads folder. Okay, let's extract like this. I right-click it and extract. On Windows 11 you may be look for more options. So, the extracted folder is here. Enter inside binary folder and copy all these 3 exe files or cut them. Move back into your C drive. Make a new folder as FFmpeg_exes or whatever the name. Do not put any space character. Okay, so this is my FFmpeg exe path. So, when I click here, I will ctrl-C or right-click and copy. I will open the environment variables one more time. Click environment variables: go to the path in system variables or user variables. Doesn't matter. If you have multiple user accounts, install it into the system variables path. Edit: click new, paste it. Alternatively, you can also click new, click browse, and in here, select its folder from C drive. FFmpeg_exe: okay, it is added, okay. By the way, the system will look starting from top to the bottom. So wherever it finds at the top, it will use it. If you have multiple Python versions, the very top will get used by default. Okay. Click okay. Click okay. Click okay. After that, open a new cmd command line and type FFmpeg and you see. We got the latest version of FFmpeg by default installed into our system and it supports all of the libraries of FFmpeg. This is the latest and biggest version of FFmpeg. Okay, we got the FFmpeg too. Now time to install CUDA libraries. To download and install CUDA libraries, you need to register a free Nvidia developer account. Let's click Windows. Actually, they changed it. Wow! Nice. So, let's click exe, and download. Previously they were asking you to log in, but looks like they changed it. Alternatively, you can also download from this link that I have uploaded into Hugging Face repository. So, from either this link or from here, you can download it. Okay, so the download has been completed. Let's open the folder, right-click, and run as administrator. If you have multiple CUDA installations, the system may get confused about which one to use, so I suggest you to uninstall previous ones and install whatever the version that you need. You can also find other CUDA versions very easily, so all you need to do is type CUDA Toolkit and archive and it will allow you to look through all of the CUDA versions as you are seeing right now. You can download and install any version that is necessary, but I suggest you to install and have CUDA 11.8 version right now. In future, it may get changed, but currently most of the AI libraries most of the AI applications are using this version, but let's see, it should still allow me to install. So, I click custom advanced click yes. Okay, everything is selected. Click next. This is where it will get installed by default: C program files, Nvidia GPU computing toolkit, CUDA and version. Okay, let's click everything and next. All right. The installation is starting. Okay, the installation has been completed. Click next and close. After that, let's open the system environment variables one more time and let's see if there is anything new. Let's open the path and now you see the CUDA is added into the very top of my system variables path. So, when any program looks for CUDA toolkit CUDA libraries, it will look into these two folders and that is very nice. Okay, it is set up. We got the CUDA installed, we got the Git installed, and now it is time to learn about git checkout, git pull, git stash, git stash pop. So I will install Automatic1111 Stable Diffusion web UI. Let's type Automatic1111 web UI. Okay, let's go to the repository, let's copy its URL and let's open back our C drive. I suggest you to install directly into your any drive, not install into documents, pictures, downloads, wherever, install into C, D, G, directly into any drive. So, let's open a new cmd in this drive. So, you see how I have opened the cmd inside this folder, then type git clone, copy-paste the URL and hit enter and it will clone the repository into your C drive. Enter inside it. Now I will start another cmd inside this drive. So, I open cmd here. Then now I am in this folder. This is where your folder is currently located. Now I can do many operations. For example, let's do git checkout dev. Now we are into the development branch and what is it you may wonder. Currently there are 14 branches of Automatic1111 web UI. When you click branches you will see that when they were last time updated. So, the last time updated development branch was five days ago. There is also Gradio 4 branch, Test fp8 branch and there is release candidate branch. When I click dev branch you will see that it is 468 commits ahead of the main branch. So, this branch has 468 improvements / fixes over the main branch. After I checked out this branch I can do git pull to verify it is latest version and it is latest version. So, this is how you update a branch. Let's return back to main branch. So, git checkout main. Okay, there is no main branch so git checkout master. Sometimes people are preferring main, sometimes they are preferring master so you see the master branch is the default branch. Now let's see the other ones. Git stash. So, when is git stash and git stash pop are useful. To be it useful you need to have a repository that is updated and you have conflicting files. To show you that I will do this. I will clone my repository. This is my own repository. By the way, you should also fork it. Star it, open notifications and if you sponsor for me I would appreciate it very much. So, let's refresh this. Okay, let's copy it, let's go back to our C drive and let's open a cmd, type git clone and copy paste the URL. For copy pasting it. I right-click. You can also alternatively right-click here and paste from here. So, let's clone it. Okay, it is getting cloned and in this repository I will change something and then I will make an update to my repository and I will do git pull. Let me show you what I mean. So, open it here. So, this is how I open git pull and you see, you see it says already up-to-date. Then I will change the readme file. Okay, for changing it, let's open it with notepad here. Okay, and in here what should we change. Strategies ok, I will type it like this. Then I will make an update to my repository. Let me do update on my computer. Okay, I did an update to repository. Let's refresh it. So, when I click this commits you see it was just now and I changed the update readme file. I just added a new keyword here. Now when I say git pull let's see what happens. Okay, it says that please commit your changes or stash them before you merge. Because there is a confliction of the files. The updated file and my modified file are conflicting. This is when it is useful to have. So, what I am going to do is I will do git stash and it will save my modified file. Then I will do git pull. You see now it is updated. The file is changed. Now when I open the readme file let's open it again with notepad. You see my changes is gone and it is updated to latest version. However, I want my modification to be back. So, what I need to do is, I will do git stash pop, and it will return back my original file, modified readme.md. So, when I open back it, I will see that it is returned back to the my version. So, you see, my 'ok' word, which I have written here, is back, and 'new' is still here. So, it pulled my change and merged it with the newest file as you are seeing right now. This is when it is useful. Okay, this is a complicated stuff, but sometimes you may be in need of that. Now, what else do we need? We have shown the git checkout, git pull, git clone, git stash. Okay, now time to how to activate a virtual environment, how to make a virtual environment, how to uninstall and install a specific library. So, for installing Stable Diffusion Automatic1111 Web UI, I will double-click and start webui-user.bat file, and this file will automatically generate a virtual environment. So, theoretically in your AI application, sometimes you may be needed to modify certain library, and in such cases, you need to update its generated virtual environment and make the changes. So, how we are going to do that? Currently, it is downloading this Torch version into our virtual environment, and it will install all of the libraries. After installation has been completed, I will change the installed Torch version. Currently, Automatic1111 Web UI installer is installing the Torch version 2.0.1. After the installation has been completed, I will activate the virtual environment folder, and I will install the Torch version 2.1, the latest version. You will see how I am going to do it. Okay, the installation has been completed. Obviously, since this is a virtual machine, and it is not a bare-metal virtual machine, it is not able to access the GPU, but it is not important. Now, I will show how to activate the virtual environment of Automatic1111 Web UI. It doesn't matter on any application that generates its own virtual environment. The procedure is the same. So enter inside scripts, this is the folder where you need to start the cmd like this: inside virtual environment, inside scripts, type activate, and now it is activated. Whatever the changes you make will only be effective in this virtual environment, not in any other of virtual environments or in your system-wide Python installation. So, let's install the latest Torch. I type PyTorch. Let's go to PyTorch website, and you see the latest version is 2.1.1. So, I copied this link, and so first I will uninstall Torch, Torch Vision, and Torch Audio. To do that, pip uninstall torch, torchvision and torchaudio and then type --yes, and it will uninstall the currently installed Torch versions. Then, I will first modify the copied command like this, and I will change the version to the specific version. If you want to see all of the versions, type like this, and then it will show you all of the available versions. Currently, these are all the available versions, as you are seeing right now. So, let's install a specific version. These are all for CUDA 11.8, you see, at the end, it shows this is for CUDA 11.8. So, which version do we want to install? So, let's install 2.1.0 and type, and it will install the Torch 2.1.0 with GPU support. Why? Because we are providing an index URL. You see, this is why this Torch installation will have GPU support. If you don't provide this index URL, it will install Torch only for CPU. It will not install it for GPU. So, this index URL has to be provided for Torch to be installed with GPU support. Alternatively, let's say you want to install an older version. So, what you need to do, let's cancel this, and let's return back to this, and change the CUDA version, like 11.3. Now, it will list me CUDA version 11.3 compatible Torch versions. You see, 1.11.0. By the way, this is also depending on your Python version. Some libraries will not be compatible with all of the Pythons. If you are not seeing a specific version that you need, then it is also meaning that your Python version is not compatible with your library version. Currently, the Python version of this virtual environment is 3.10.11. Okay. So how can you also compose a virtual environment? Let's open a folder inside here. Let's open a new cmd, so I will type Python -m venv and the folder name of venv venv. Okay, it is now vevnv, and you see, it is generated. This will be generated with your default Python. Let's say my default Python is 3.10.11. What if you want to have multiple Pythons? Okay, let's also download Python 3.11. You can't have multiple Pythons of the same version like Python 3.10.5 or Python 3.10.7, but you can have multiple different major versions. So, this is Python 3.11, and from here, let's download the Windows installer. Okay, let's install it. I will not add it to the path, so it will not conflict with my 3.10. Let's customize next. Okay, install everything. I am not adding it to the environment variables. This is important. Otherwise, I have to change the environment variable order. Okay, this is 10.11 like this: Okay, install. Okay, the installation has been completed. Let's open our environment variables. From here, let's look at the environment variables, and we will not see that Python in our path. You see, Python 3.11 is not here. So, when I type cmd and type Python, I should still get 3.10.11, not 3.11. So, how can I utilize Python 11 version? Enter inside Python 11, right-click the exe file while keep hitting the shift key, and you see there is Copy as path. Copy it, and let's enter inside this folder, type cmd. This time, I will copy-paste the path of the specific Python version like this. You see, it also has a quotation mark, and I will type -m venv, and let's say 3.11 venv like this, or let's say venv 3.11, and hit enter, and it will generate a virtual environment based on this Python version installation. Now, let's move into a new folder. For moving into folders, type cd, type venv, hit tab, and it will auto-complete the name like this. You see, I am hitting the tab key, then move into the scripts, type activate, and let's type Python --version, and you see the virtual environment. Python version is 3.11, or let's say, just type Python, and now it is also 3.11 as you are seeing. You can also change your default Python version. Let's open the environment variables, click environment variables, click edit path and in here, click new, then click browse. Select the Python 11 from here, 3.11, then also click new, and click browse, and then go to the 3.11, and in here, you need to select scripts. Okay. You see, now we have two of them. Let's move this to the up version like this, this, and these two, and will it be okay, will it be sufficient? Let's see. Python. And no, still, 3.10 is default. Why? Because the order of the environment path variables matters. So, I edit. I will move these to the very top like this, then also move this to the very top like this. Okay, click. Okay, okay, then open a new cmd. You need to open a new one. Type Python. And now, my main Python. My default Python is 3.11. This is how you can set up multiple Pythons. This is how you can add, modify, change, path, change your default applications. This is also the same for runpodctl. Runpodctl is the utility that we use for RunPod. If you are following my RunPod tutorials, you will know that. So, we need to first download the runpodctl. How am I going to do that? So, let's copy this. Let's open a new folder inside C. Let's say runpodctl_exe. Open a new cmd inside this folder. Copy-paste the link. Okay, if there is no wget on Windows. So, we need to just download this file from here actually. It will download a file like this. Okay, let's say download suspicious file. Let's open it, and I will cut it. Return back into the runpodctl_exe. Let's name it as runpodctl.exe like this. Let's open a new environment variable. Currently, when I type cmd and type runpodctl, you see, it is not recognized. So, I go to environment variables. I go to path, edit, click new, and click browse. Let's select the runpodctl from here. Okay, okay okay okay. Let's open a new CMD. And when I now type runpodctl, you see, now it is by default. So now, I can use runpodctl. This is the logic of pathing in Windows. This is super important to learn and understand. I will update this file as it be necessary. The link to this file will be in the description of the video and also in the comment section of the video. I appreciate it very much if you support me on Patreon, when you click this link, you will see our Patreon page. I am sharing a lot of useful scripts here with guidance, written tutorials, instructions. You see I have 75 sharing. We have over 3400 members, but not all of them are paid members. You can also join for free, but if you join for free, you will not see the subscription-based scripts, only the free ones. You can buy me a coffee and support me. I would appreciate that very much. We have two supporters so far. You can follow me on Medium. I am writing a lot of articles, sharing super important stuff here. You can follow me on CivitAI. I am writing a lot of articles on CivitAI as well. You can follow me on DeviantArt. You should subscribe to our channel. You can follow me on LinkedIn. I have over 3000 followers. I am sharing a lot of useful information here as well. You can also purchase our Udemy course, but I suggest you become a Patreon member and follow our YouTube channel. That is better. But if you don't have any other options, you can purchase my Udemy course and access the scripts there. However, Patreon is the most up-to-date way, and you can follow me on Twitter. I hope you have enjoyed this tutorial, hopefully much more amazing tutorials are coming and hopefully, they will help you significantly. For example, I am going to make a tutorial for Magic Animate Automatic Installer and how to use it. This tutorial was mandatory for that tutorial because you had to install C++ tools and FFmpeg. This is why I recorded this tutorial previously, but this is a major tutorial. This tutorial will help you significantly if you are interested in AI overall. Thank you very much. See you later hopefully.
Info
Channel: SECourses
Views: 3,893
Rating: undefined out of 5
Keywords: Pillars of AI, AI Application Libraries, Install Tutorial, Generative AI, Stable Diffusion, Large Language Models, Web UI, Voice Cloning, Text-to-Speech, Image-to-Image, Text-to-Image, Text-to-Video, Video-to-Video, Image-to-Video, AI Tutorial, Python Installation, Python 3.10.11, Default Python Version, Roop Deep Fake Application, FFmpeg Installation, CUDA Libraries, Git Installation, Git Large, Git Checkout, Git Pull, Git Clone, Git Stash, Git Stash Pop
Id: -NjNy7afOQ0
Channel Id: undefined
Length: 34min 25sec (2065 seconds)
Published: Fri Dec 15 2023
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.