moving to debian + docker as proxmox replacement (part 2)
it’s been some time since i decided to migrate from proxmox + lxc containers to debian + docker containers and it has been quite the learning curve. i figured i’d go ahead and start off with the method i am going to use to migrate over.
first. i installed debian (trixie was the latest stable at the time of this writing but debian is always stable and just works). then i used these instructions in order to install the docker repo because the debian repo always has older docker packages.
like most people i figured i’d just run docker as root on the bare metal and let it do its thing. docker runs containers so it’s safe right? probably, but my brain was just screaming at how it runs some of these containers as root using a daemon that runs as root. in a perfect world i’d be using virtual machines (VMs) for this. unfortunately i need the use of my intel a380 gpu across the board in multiple containers that i don’t want to lump together into a pile of containers running as one user (root). my a380 DOES NOT support sr-iov (which lets you share gpu slices across VMs thus allowing you to use a single gpu for multiple VMs). better models like the pro b50 can, but i’m not spending $600 when i have a perfectly capable gpu for my needs.
my solution: use docker in rootless mode and create users that i’d treat like VMs. just a note. the reasoning i had for this is that if a container is compromised then (hopefully) the damage will be contained (no pun) as a user and not as root on the host. granted, the possibility of some l33t 0day wrecking my host is low, but that doesn’t mean i shouldn’t do it. another reason is that it allows for organizational and use case structuring. i can make a user called media and put things like jellyfin, plex, dispatcharr in it; i can make a user called admin and have things like freeipa running in it. i’d honestly rather run these in VMs and use sr-iov to give the VMs gpu slices, but, like most people, i’m too poor for that.
so it would probably be structured something like this with “users” being listed as what runs what.
host -------------------|------------------------
admin media network
| | |
freeipa dispatcharr znc
debian jellyfin mastodon
plex continuwuation
this was just a quick example. for the admin “user” i want to have something akin to a debian “shell box” for doing shell stuff. in proxmox i’d just use lxc, make a debian container, and call it a day treating it similar to how i would as a VM that’s not a VM. unfortunately i haven’t found a direct way to do this with docker and will come back to that later. i sorted out most of it but it runs similar to nixos.
the thing is that i want the debian “shell box” (running as the “user” admin) to share the same gpu as dispatcharr, jellyfin, and plex (running as the media “user”). i know mastodon also does a lot of ffmpeg tomfoolery for videos and photos from its feed but i haven’t looked into that just yet. if i can get that to use the gpu as well then all the better.
so that is the goal and i think i sorted out most of how i want it structured and managed. i’ll be doing most of the work on the host using docker compose. i guess the new standard is to use compose.yml files in the project directories so i’ll take that route.