Virtual environments — Python: Setting up the environment

• Designing virtual environments
• Creating a virtual environment
• Activating the environment

You already know that pip installs packages in one of two environments:

• The system-wide environment
• The user environment

When importing a module or package, the Python interpreter looks for it first in the user environment, then in the system-wide environment. This sequence allows us to use the exact versions of the libraries and Python programs we need.

But even two environments aren't enough when a programmer starts working on several projects because projects can have different dependencies.

And there are even more complex cases. Different projects may depend on one library but require different versions of it. That'll mean a version conflict.

A Python developer needs some mechanism to keep different projects in isolated sandboxes. There is a mechanism precisely for this. It's called virtual environments.

Designing virtual environments

Each virtual environment is a directory.

Its contents are structurally similar to the system-wide environment — the subdirectories we named and populated accordingly. Let's look at an example:

tree env

env
├── bin
│   ├── activate
│   …
│   ├── pip
│   …
│   └── python3 -> …/python3
├── lib
│   └── python3.6
│       └── site-packages
│           ├── pip
│           │   ├── …
│           …
│           └── setuptools-40.6.2.dist-info
│               ├── …
…

The bin/ directory locates in the virtual environment directory. In the bin/ directory, you can find:

• A copy of the interpreter named python3 (symbolic link to the original)
• A copy of the pip executable pip

There are libraries already installed in the environment in the adjacent directory at lib/python3.6/site-packages. As a rule, a newly created environment has:

• The pip package installed (the bin/pip — the executable is its entry point)
• The Setuptools package

These two packages are the minimum for developing a project in Python.

When working in the environment, you shouldn't run the system Python and pip, but rather the executables from the bin directory. When the Python interpreter is in the environment, it knows where all available packages are. The interpreter finds them in the relative path ../lib/python3.6. A copy of pip from the bin/ directory installs packages in the same environment without affecting the system. It is the isolation we talked about at the beginning of the lesson.

Creating a virtual environment

There is no need to manually create the entire hierarchy of directories and files described above. There is a module called venv for that.

On macOS and Windows, this module is in a bundle with Python. On Ubuntu, you have to install it separately with a command:

sudo apt install python3-venv

Check that the module is installed and ready to use:

python3 -m venv --help

usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear] [--upgrade] [--without-pip] [--prompt PROMPT] ENV_DIR [ENV_DIR ...]

Usually, we create the environment with the command python3 -m venv name_environment. Let's try to create a virtual environment and install the cowsay package there:

We installed the package with an entry point, which we can call with the command first_venv/bin/cowsay. Also, the package becomes available to the interpreter we started in the environment. In this form, we can fully use the virtual environment. But constantly entering commands with the prefix first_venv/bin/ is inconvenient.

There is a way to simplify calling the commands available in the environment. It's called activation.

Activating the environment

When we created the environment, we placed a shell script in the bin subdirectory. We call it:

• activate in macOS and Ubuntu
• activate.bat in Windows

To execute this script, you need to call this command:

• In macOS and Ubuntu:

  source first_venv/bin/activate
  

• In Windows:

  C:\> first_venv\Scripts\activate.bat
  

In the command above, the subdirectory with the executable files isn't called bin but Scripts.

In Ubuntu, the activation is as follows:

After activation, you no longer need to specify the path to the called executable. Now Cowsay and Python are called without the prefix, but they're still the same commands from the environment.

The shell prompt has also changed to display the name of the environment. It works on macOS and Ubuntu and reminds us that we're in a virtual environment.

We can deactivate the environment with the deactivate command, which becomes available after activation.

Activation and deactivation only affect the current session, i.e., they're only visible in this terminal. It is convenient because you can have several environments and activate them simultaneously in different terminal windows this way.

Virtual environments are a powerful and convenient tool for isolating programs from each other and the system. Isolation even allows you to use different versions of Python in different environments. When working on projects of various ages, this is often vital! Therefore, as usual, we recommend adding virtual environments to the list of topics for more in-depth study in the future.

Recommended materials

1. Creation of virtual environments (venv)
2. The pipx project