Crazy Macer Life

i want to use jellyfin and come away from using plex. there is a quick and easy post on how to get jellyfin working in a jail here but it doesn’t really speak about how to get the hw accel trancoding working. the freebsd server has an ancient nvidia P400 in it which worked fine in proxmox but now i have to sort out how to get it working in freebsd AND inside of a jail.

i followed the handbook and installed nvidia-drm-kmod to see what happens. i can see the card in nvidia-smi but it doesn’t seem to want to do anything in a jail. i’m not sure if this is the right approach for this considering i am not using it for xorg or wayland but for transcoding ONLY in a jail. i also found that trying to set the card in persistent mode with nvidia-smi results in the server rebooting. that’s not good. i’m still working on this.

so i gave it quite a few hours and really tried to get it going. right now it does not seem like it’s possible to get this working and all the pkgs are meant for running an nvidia on a desktop. i couldn’t even get ffmpeg to transcode on the host manually. i’m giving up for now.

got back to it. seems like the missing piece was libc6-shim and nv-sglrun to get it working on the host. lets see if i can move that over to the jail after some testing.

this was one of those things where i did so much trying to fix it that i can’t even remember how to back track to explain it. I’ll go ahead and show the highlights.

root@devil:~ # pkg install nvidia-driver nvidia-drm-kmod libc6-shim
...
root@devil:~ # nano -w /boot/loader.conf

fusefs_load="YES"
fdescfs_load="YES"
linprocfs_load="YES"
linsysfs_load="YES"
linux_load="YES"
lindev_load="YES"
tmpfs_load="YES"
hw.nvidia.registry.EnableGpuFirmware=1
hw.nvidiadrm.modeset=1

root@devil:~ # nano -w /etc/devfs.rules

[devfsrules_jail_gpu=128]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path fuse unhide
add path zfs unhide
add path 'nvidia*' unhide
add path 'dri*' unhide
add path 'drm*' unhide

root@devil:~ # nano -w /usr/local/bastille/jails/fin/jail.conf 

fin {
  enforce_statfs = 1;
  devfs_ruleset = 128;
  exec.clean;
  exec.consolelog = /usr/local/bastille/logs/fin_console.log;
  exec.start = '/bin/sh /etc/rc';
  exec.stop = '/bin/sh /etc/rc.shutdown';
  host.hostname = fin;
  mount.devfs;
  mount.fstab = /usr/local/bastille/jails/fin/fstab;
  allow.mount;
  allow.mount.fdescfs;
  allow.mount.fusefs;
  allow.mount.tmpfs;

  path = /usr/local/bastille/jails/fin/root;
  securelevel = 2;
  osrelease = 15.0-RELEASE-p1;

  vnet;
  vnet.interface = e0b_fin;
  exec.prestart += "jib addm fin lagg0";
  exec.prestart += "ifconfig e0a_fin description \"vnet0 host interface for Bastille jail fin\"";
  exec.poststop += "ifconfig e0a_fin destroy";
  allow.mlock = 1;
}

IN THE JAIL

root@fin:~ # pkg install libc6-shim
root@fin:~ # nv-sglrun ffmpeg ...

that works! but i can’t figure out how to get jellyfin to call nv-sglrun when running ffmpeg.

ok. with some help from #freebsd@libera i sorted this out. it’s a really ugly way to do it.

root@fin:/usr/local/bin # mv ffmpeg ffmpeg.bin

root@fin:/usr/local/bin # nano -w ffmpeg

#!/bin/sh
nv-sglrun ffmpeg.bin "$@"

after doing this jellyfin should use ‘nv-sglrun ffmpeg.bin “filename –flags…” to play the files. i tested it and it’s working with the nvidia! the only problem is what happens when there is an ffmpeg update? i guess i’ll have to keep an eye on it and maybe ask the pkg maintainer to make a different dependency.

i noticed that it wasn’t obeying the setting to use nvdec in transcoder settings in jellyfin for some reason. i had to modify the script a little bit. like i said this is pretty ugly but it gets the job done. now the P400 is doing maximum transcoding.

#!/bin/sh
nv-sglrun ffmpeg.bin -hwaccel cuda "$@"

that should make it use nvdec on whatever the input is. i noticed my CPUs were getting hammered from the decoding side.