VF Assigned More Than Once

[spetrillo]

I am leaving the above post, even though I really could have deleted it. I firmly believe that you need to walk away from something when you cannot seem to figure it out. Trying to plow through can lead to more problems and you end up getting really frustrated. So after I posted above I walked away until now. I was just finishing dinner when it came to me. OPNsense can set LAGGs within its network config process....duhh! I am redoing my OPNsense install and indeed yes I now have 5 vlans that will run across one LAGG. It was staring me right in the face but I was too frustrated to see it.

Install is going well and I will get all my interfaces set. I am then going to stop. I will wake up early tomorrow, so I can shutdown my current physical firewall and activate the new virtual.

I do have one question I hope someone could respond to. I am at the point of beginning to use my vf interfaces for my other vms. As mentioned 3 of the 4 physical interfaces are in a LACP LAGG. Does that mean this config trickles down to the vfs or could I use the Linux bridge that I setup, that calls the Linux bond of the 3 interfaces. Not sure how to allocate the vfs. I am also starting to think I might not need the vfs, since I was able to pass through the physical interfaces and use them as PCIe devices in my OPNsense.

I think I figured out my last paragraph from the last post. Since I am using PCIe passthrough and SR-IOV there is no reason to define any network settings within Proxmox, with the exception of the IP for the Proxmox server itself. If am going to use SR-IOV I simply pass the PCI device, whether physical or virtual, straight to the VM being built...again another duhh moment.

I have reconfigured my Proxmox server, so the new network config is attached. I have 4 vfs defined per physical port on my I350. You will see vmbr1 and vmbr1.1. That is for the IP of the Proxmox server. From the switch perspective port 1 is dedicated to vlan 1 only, which is my mgmt vlan. Ports 2-4 is a LACP LAGG, for vlans 10,12,20,25,30.

Now I have to learn how to build Ubuntu with VLANs. I do not remember if there was a place in the network config of the install process to define vlan over IP. I could always define the IP for vlan 1 and then move it to the LAGG port when I have vlan support installed on Ubuntu. It looks like vlan support is an add-in, and not part of the base install process. That's the new question to answer. Given my track record maybe I should just walk away now and come back?!?!

For those of you that are running ZFS and systemd attached is my /etc/modules and /etc/kernel/cmdline config! If you need anything else please let me know!

 

[Spaneta]

Nice to see you evolution across this journey !
I will clearly come back here to re-read when I begin SR-IOV lab myself.
Thanks a lot for your feedback.