Updates for Debian 12 - Dealing with Pip changes
So it's been a little while since I've tinkered with this stuff, so I've been self-refreshing myself as I go along here.
The big catalyst for this work was the release of Proxmox 8. The key takeaway during the upgrade is that it's a typical Debian upgrade, and so it went more or less without incident. Great!
The things that hang off Proxmox
While the upgrade to Proxmox went fine, there's a bunch of stuff that "hangs off" Proxmox: A load balancer, a playbook to create cloud images, a backup system, and so on. Some of these are one-off application installs, or single-purpose VMs that do a specific thing, but in other cases, they're stuff that I've written.
Thinking about the playbook that creates cloud images, I have a couple of tasks that look like this:
1 - name: Install pip 2 ansible.builtin.apt: 3 name: 4 - pip 5 state: present 6 7 - name: Install Proxmoxer on the machine 8 ansible.builtin.pip: 9 name: proxmoxer 10 state: present
With a working Debian system, we install
pip, then using
pip, I'm installing the
proxmoxer packages so that I can manage Proxmox from Python.
Under Debian 12, we don't like installing packages into the global space anymore (resonable..). Ansible will return the error from the shell:
1error: externally-managed-environment 2 3× This environment is externally managed 4╰─> To install Python packages system-wide, try apt install 5 python3-xyz, where xyz is the package you are trying to 6 install. 7 8 If you wish to install a non-Debian-packaged Python package, 9 create a virtual environment using python3 -m venv path/to/venv. 10 Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make 11 sure you have python3-full installed. 12 13 If you wish to install a non-Debian packaged Python application, 14 it may be easiest to use pipx install xyz, which will manage a 15 virtual environment for you. Make sure you have pipx installed. 16 17 See /usr/share/doc/python3.11/README.venv for more information.
So, as the error suggests, I can either:
- Install an equivalent package from apt:
- Create a virtualenv and install proxmoxer there.
In the case of the script to create the cloud images, I chose the former: I don't actually write a script and put it anywhere on the target host - it's being leveraged from an Ansible playbook.
In the case of the loadbalancer, I am using a script from https://github.com/cvk98/Proxmox-load-balancer. This comes with it's own set of pip requirements.txt that I can, in fact, set up a virtualenv.
I think the approach is now:
apt, along with
pipand anything else you want.
- Clone the source repo into a new directory
- Create a virtualenv with the following task:
1 - name: Create a virtualenv with the downloaded requirements 2 ansible.builtin.pip: 3 requirements: /src/proxmox-loadbalancer-src/requirements.txt 4 virtualenv: /src/proxmox-loadbalancer
- Copy the app's configuration file (I have this as an Ansible template)
- Update the app's systemd config from the git repo so that it'll launch python from the virtualenv:
1[Unit] 2Description=Proxmor cluster load-balancer Service 3After=network.target 4 5[Service] 6Type=simple 7User=root 8NoNewPrivileges=yes 9ExecStart=/src/proxmox-loadbalancer/bin/python3 /src/proxmox-loadbalancer-src/plb.py 10WorkingDirectory=/src/proxmox-loadbalancer/ 11Restart=always 12RestartSec=300 13 14[Install] 15WantedBy=multi-user.target
- Enable the newly minted
- Start the
Did it blend?
The usual tools can be used to test that it's working:
1root@loadbalancer:~# systemctl status load-balancer.service 2* load-balancer.service - Proxmor cluster load-balancer Service 3 Loaded: loaded (/etc/systemd/system/load-balancer.service; enabled; preset: enabled) 4 Active: active (running) since Mon 2023-06-26 16:56:25 MDT; 2min 37s ago 5 Main PID: 14464 (python3) 6 Tasks: 1 (limit: 76663) 7 Memory: 20.3M 8 CPU: 169ms 9 CGroup: /system.slice/load-balancer.service 10 `-14464 /src/proxmox-loadbalancer/bin/python3 /src/proxmox-loadbalancer-src/plb.py 11 12Jun 26 16:56:25 loadbalancer systemd: Stopped load-balancer.service - Proxmor cluster load-balancer Service. 13Jun 26 16:56:25 loadbalancer systemd: Started load-balancer.service - Proxmor cluster load-balancer Service. 14Jun 26 16:56:26 loadbalancer python3: INFO | START ***Load-balancer!*** 15Jun 26 16:56:26 loadbalancer python3: INFO | Need to balance: False 16Jun 26 16:56:26 loadbalancer python3: INFO | The cluster is balanced. Waiting 300 seconds.
Monitor the logs:
1root@loadbalancer:~# journalctl -u load* -f 2Jun 26 16:51:25 loadbalancer systemd: Started load-balancer.service - Proxmor cluster load-balancer Service. 3Jun 26 16:56:25 loadbalancer systemd: Started load-balancer.service - Proxmor cluster load-balancer Service. 4Jun 26 16:56:26 loadbalancer python3: INFO | START ***Load-balancer!*** 5Jun 26 16:56:26 loadbalancer python3: INFO | Need to balance: False 6Jun 26 16:56:26 loadbalancer python3: INFO | The cluster is balanced. Waiting 300 seconds. 7Jun 26 17:01:26 loadbalancer python3: INFO | Need to balance: False 8Jun 26 17:01:26 loadbalancer python3: INFO | The cluster is balanced. Waiting 300 seconds.