Python virtual environments

Python virtual environments are useful as it provides a closed system that the user has almost all control in installing packages with pip. Most of the time, commonly used packages like scipy, numpy and matplotlib can be installed without issues. Some of the time, a OS level dependency will create some problems. In this case, contact us.

In the example below we log in to gamma, and we set up an environment called testenv. Then we activate it. pip list shows you that the only two available packages are pip and setuptools. Now you are free to install packages.

user@gamma ~ $ python3 -m venv testenv 
user@gamma ~ $ source testenv/bin/activate 
(testenv) user@gamma ~ $ pip list 
Package    Version 
---------- ------- 
pip        21.3.1 
setuptools 53.0.0

A more recommended way to install packages is to keep a requirements.txt file which contain on each line the name and version of the package you would like to install. For example

matplotlib==3.10.1
numpy>=2.2.4

You can also then activate the Python virtual environment within your slurm scripts, as they should have access to your /user partition.

References

https://docs.python.org/3/library/venv.html

-- MarkWong - 21 Mar 2025