Updates for Debian 12 - Dealing with Pip changes

calendar Jun 26, 2023 · 4 min read · debian ansible

Stuff

So it's been a little while since I've tinkered with this stuff, so I've been self-refreshing myself as I go along here.

The big catalyst for this work was the release of Proxmox 8. The key takeaway during the upgrade is that it's a typical Debian upgrade, and so it went more or less without incident. Great!

The things that hang off Proxmox

While the upgrade to Proxmox went fine, there's a bunch of stuff that "hangs off" Proxmox: A load balancer, a playbook to create cloud images, a backup system, and so on. Some of these are one-off application installs, or single-purpose VMs that do a specific thing, but in other cases, they're stuff that I've written.

Python's PIP

Thinking about the playbook that creates cloud images, I have a couple of tasks that look like this:

 1    - name: Install pip
 2      ansible.builtin.apt:
 3        name:
 4          - pip
 5        state: present
 6
 7    - name: Install Proxmoxer on the machine
 8      ansible.builtin.pip:
 9        name: proxmoxer
10        state: present

With a working Debian system, we install pip, then using pip, I'm installing the proxmoxer packages so that I can manage Proxmox from Python.

Under Debian 12, we don't like installing packages into the global space anymore (resonable..). Ansible will return the error from the shell:

 1error: externally-managed-environment
 2
 3× This environment is externally managed
 4╰─> To install Python packages system-wide, try apt install
 5    python3-xyz, where xyz is the package you are trying to
 6    install.
 7
 8    If you wish to install a non-Debian-packaged Python package,
 9    create a virtual environment using python3 -m venv path/to/venv.
10    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
11    sure you have python3-full installed.
12
13    If you wish to install a non-Debian packaged Python application,
14    it may be easiest to use pipx install xyz, which will manage a
15    virtual environment for you. Make sure you have pipx installed.
16
17    See /usr/share/doc/python3.11/README.venv for more information.

So, as the error suggests, I can either:

  • Install an equivalent package from apt: python3-proxmoxer
  • Create a virtualenv and install proxmoxer there.

In the case of the script to create the cloud images, I chose the former: I don't actually write a script and put it anywhere on the target host - it's being leveraged from an Ansible playbook.

In the case of the loadbalancer, I am using a script from https://github.com/cvk98/Proxmox-load-balancer. This comes with it's own set of pip requirements.txt that I can, in fact, set up a virtualenv.

I think the approach is now:

  • Install python3-virtualenv from apt, along with pip and anything else you want.
  • Clone the source repo into a new directory
  • Create a virtualenv with the following task:
1    - name: Create a virtualenv with the downloaded requirements
2      ansible.builtin.pip:
3        requirements: /src/proxmox-loadbalancer-src/requirements.txt
4        virtualenv: /src/proxmox-loadbalancer
  • Copy the app's configuration file (I have this as an Ansible template)
  • Update the app's systemd config from the git repo so that it'll launch python from the virtualenv:
 1[Unit]
 2Description=Proxmor cluster load-balancer Service
 3After=network.target
 4
 5[Service]
 6Type=simple
 7User=root
 8NoNewPrivileges=yes
 9ExecStart=/src/proxmox-loadbalancer/bin/python3 /src/proxmox-loadbalancer-src/plb.py
10WorkingDirectory=/src/proxmox-loadbalancer/
11Restart=always
12RestartSec=300
13
14[Install]
15WantedBy=multi-user.target
  • Reload systemd
  • Enable the newly minted load-balancer service
  • Start the load-balancer service

Did it blend?

The usual tools can be used to test that it's working:

 1root@loadbalancer:~# systemctl status load-balancer.service
 2* load-balancer.service - Proxmor cluster load-balancer Service
 3     Loaded: loaded (/etc/systemd/system/load-balancer.service; enabled; preset: enabled)
 4     Active: active (running) since Mon 2023-06-26 16:56:25 MDT; 2min 37s ago
 5   Main PID: 14464 (python3)
 6      Tasks: 1 (limit: 76663)
 7     Memory: 20.3M
 8        CPU: 169ms
 9     CGroup: /system.slice/load-balancer.service
10             `-14464 /src/proxmox-loadbalancer/bin/python3 /src/proxmox-loadbalancer-src/plb.py
11
12Jun 26 16:56:25 loadbalancer systemd[1]: Stopped load-balancer.service - Proxmor cluster load-balancer Service.
13Jun 26 16:56:25 loadbalancer systemd[1]: Started load-balancer.service - Proxmor cluster load-balancer Service.
14Jun 26 16:56:26 loadbalancer python3[14464]: INFO | START ***Load-balancer!***
15Jun 26 16:56:26 loadbalancer python3[14464]: INFO | Need to balance: False
16Jun 26 16:56:26 loadbalancer python3[14464]: INFO | The cluster is balanced. Waiting 300 seconds.

Monitor the logs:

1root@loadbalancer:~# journalctl -u load* -f
2Jun 26 16:51:25 loadbalancer systemd[1]: Started load-balancer.service - Proxmor cluster load-balancer Service.
3Jun 26 16:56:25 loadbalancer systemd[1]: Started load-balancer.service - Proxmor cluster load-balancer Service.
4Jun 26 16:56:26 loadbalancer python3[14464]: INFO | START ***Load-balancer!***
5Jun 26 16:56:26 loadbalancer python3[14464]: INFO | Need to balance: False
6Jun 26 16:56:26 loadbalancer python3[14464]: INFO | The cluster is balanced. Waiting 300 seconds.
7Jun 26 17:01:26 loadbalancer python3[14464]: INFO | Need to balance: False
8Jun 26 17:01:26 loadbalancer python3[14464]: INFO | The cluster is balanced. Waiting 300 seconds.