YUFENG GUO: Every data scientist
has different preferences when it comes to their
programming environment-- vim versus emacs, tabs
versus spaces, Virtualenv versus Anaconda. Today I want to share
with you my environment for working with data and
doing machine learning. You most definitely do
not need to copy my setup, but perhaps some bits of it
can serve as useful inspiration for your development
environment. To start with, we need
to talk about Pip. Pip is Python's package manager. It has come built into
Python for quite a while now, so if you have Python, you
likely have Pip already. Pip installs packages
like tensorflow and numpy, pandas and Jupyter,
and many, many more, along with their dependencies. Many Python resources
are delivered in some form of Pip packages. Sometimes you may see a
file called requirements.txt in someone's folder
of Python scripts. Typically, that file outlines
all of the Pip packages that that project
uses, so you can easily install everything
needed by using pip install -r requirements.txt. As part of this
ecosystem, there's a whole world of version
numbers and dependencies. I sometimes need to
use different versions of a given library for different
projects that I'm working on. So I need a way to organize
my groups of packages into different
isolated environments. There are two popular
options currently for taking care of managing
your different Pip packages-- virtualenv and anaconda. Virtualenv is a
package that allows you to create named
virtual environments where you can install Pip packages
in an isolated manner. This tool is great
if you want to have detailed control
over which packages you install for each
environment you create. For example, you could create an
environment for web development with one set of libraries,
and a different environment for data science. This way, you won't need to have
unrelated libraries interacting with each other, and it allows
you to create environments dedicated to specific purposes. Now, if you're primarily
doing data science work, Anaconda is also a great option. Anaconda is created by
Continuum Analytics, and it is a Python
distribution that comes preinstalled with lots
of useful Python libraries for data science. Anaconda is popular
because it brings many of the tools used in
data science and machine learning with just
one install, so it's great for having a
short and simple setup. Like Virtualenv, Anaconda
also uses the concept of creating environments so as
to isolate different libraries and versions. Anaconda also introduces its
own package manager called conda from where you can
install libraries. Additionally, Anaconda still has
the useful interaction with Pip that allows you to install
any additional libraries which are not available in the
Anaconda package manager. So-- which one do I use,
virtualenv or anaconda? Well, I often find
myself testing out new versions of tensorflow
and other libraries across both Python
2 and Python 3. So ideally, I would like
to be able to try out different libraries on both
virtualenv and anaconda, but sometimes those
two package managers don't necessarily play nicely
with each other on one system. So I have opted to use both,
but I manage the whole thing using a library called pyenv. Conceptually, pyenv sits atop
both virtualenv and Anaconda and it can be used to control
not only which virtualenv environment or Anaconda
environment is in use, but it also easily
controls whether I'm running Python 2 or Python 3. One final aspect of
pyenv that I enjoy is the ability to set a
default environment for a given directory. This causes that
desired environment to be automatically activated
when I enter a directory. I find this to be way
easier than trying to remember which
environment I want to use every time I work on a project. So which package
manager do you use? It really comes down to your
workflow and preferences. If you typically just use
the core data science tools and are not concerned with
having some extra libraries installed that you
don't use, Anaconda can be a great choice, since
it leads to a simpler workflow for your needs and preferences. But if you are someone who loves
to customize your environment and make it exactly
like how you want it, then perhaps something like
virtualenv or even pyenv maybe more to your liking. There's no one right way
to manage Python libraries, and there's certainly more
out there than the options that I just presented. As different tools
come and go, it's important to remember that
everyone has different needs and preferences. So choose for yourself-- what
tools out there serve you best? So what does your
Python moment look like, and how do you keep it
from getting out of control? Share your setup in
the comments below. Thanks for watching this
episode of Cloud AI Adventures. Be sure to subscribe
to the channel to catch future episodes
as they come out.
