Crazy Macer Life

i have 3 nics in this thing. 2 onboard 1gbit nics and 1 10gbit nic. i’ve bumped into an issue before where the 10gbit died and didn’t have failover configured (overheated or whatever). in proxmox it’s called bonding, in freebsd it’s called lagg (i think for everything it is called lagg but whatever linux).

get out of the mindset that interfaces all need ips. they are just ports. the os handles the packet routing. step one for me is to take all 3 nics and add them to a lagg for failover in case one dies. make sure you can access the server (physically or with ipmi) because you can break things if you’re using ssh.

the two onboard nics are identified by freebsd as igb0 and igb1. the 10gbit nic is identified as ix0. you can see your interfaces using ifconfig -a. they are all plugged into switches and active. so it’s a sort of two step process. one is to create lagg0 for the failover. two is to create a bridge that has a local ip and uses lagg0 as its interface. you can do a lot more with this stuff but this is how i did it:

# nano -w /etc/rc.conf

ifconfig_ix0="up"
ifconfig_igb0="up"
ifconfig_igb1="up"
cloned_interfaces="bridge0 lagg0"
ifconfig_lagg0="laggproto failover laggport ix0 laggport igb0 laggport igb1"
ifconfig_bridge0="addm lagg0"
ipv4_addrs_bridge0="192.168.1.12/24"
defaultrouter="192.168.1.1"

a quick rundown. i bring all 3 interfaces up (they’re all plugged in). i am going to make 2 cloned interfaces. the lagg0 interface and the bridge0 interface. i configure lagg0 to use all 3 interfaces for failover. the first interface ix0 is the 10gbit interface so that is the one i prefer gets used. it will failover in the order you add them to that line. so adding the laggport as ix0 first makes it the MASTER and the others are the SLAVE interfaces for the lagg. if the 10gbit starts on fire then in theory the lagg will move onto the first 1gbit nic (igb0) and if that one fails too then igb1. next i configure bridge0 using lagg0. i then assign the ip TO THE BRIDGE (this is kind of important since the jail interfaces will also be attached to this bridge later). lastly… my router as the default.

my setup is pretty basic and you have other options if you want to boost bandwidth withthe likes of lacp or whatever. just note that your switch has to support those sorts of things.