Migrating from anaconda to venv

I recently learned at work that using Anaconda for commercial product development work is no longer free for non-small organizations. Apparently this has been the case since 2020 (which I am still mentally in). There are more details on this reddit thread.

I quite like anaconda and have been a consistent user since I started learning python. I had briefly explored venv for creating virtual environments but anaconda felt more convenient. The license change forced us to move en masse to venv. What follows is a set of instructions to install and start coding using venvs on machines where you don’t want to fiddle with the system python.

In conda it is possible to create an environment with any python version. With venv, we can, by default, only create environments with the installed system version of python that I could not change without sudo.

• Download and compile the required python

  • Download the python source tarball and compile it. I started with the python 3.9.16 here.
  • Extract $> tar -xvf downloaded_python.tar.gz. This will create a Python3.9.16 folder in that directory.
  • I was more interested in getting pytorch working with GPUs, so we need to compile python with several libraries with sudo apt install libssl-dev libncurses5-dev libsqlite3-dev libreadline-dev libtk8.6 libgdm-dev libdb4o-cil-dev libpcap-dev libffi-dev
  • Compiling: $> ./configure, $> sudo make altinstall. At the end of which, you should see a python executable in the folder.

• Installing venv

  • Deactivate conda if you had it installed. $> conda deactivate
  • Create a virtual environment with the newly available python3.9 executable python3.9 -m venv /folder/of/new_env
  • Activate it: $> source /folder/of/new_env/bin/activate
  • The prompt will now look like (new_env)$>, similar to conda.
  • pip install anything you need

• Cleaning up anaconda

  • $> conda install anaconda-clean. Run $> anaconda-clean and finally delete the anaconda installation folder $> rm -rf /path/anaconda_installation