Backup server local setup

[DigiDoc]

I am not sure this is the place for my questions. I have virtualized my PBS on Synology Virtualization Manager. This is connected to PVE instance through my virtualized router (OPNsense) that is running on the PVE host. This setup works fine until I have my router VM goes down then I loose all access to my network and I am unable to restore those backups. What is a good solution for my network setup?

 

[Dunuin]

I'm basically doing the same. PBS as a VM on my TrueNAS server and OPNsense VM on my PVE server. The difference is that I run two OPNsense VMs in master-backup Mode so they are highly available. Both OPNsense VMs will run in parallel and will be kept in sync using pfsync but only the master OPNsense will actively be used. As soon as the master OPNsense goes down, within a second the backup OPNsense will become the new master OPNsense. And the backup OPNsense is running on my TrueNAS server. So no matter what of the two servers will fail, there is always a working OPNsense VM. Also nice because a reboot of the PVE server or "stop" mode backup of the OPNsense VM won't cut my internet connection.

But OPNsense needs interfaces with the identical names for HA to work. Not sure if you can do that with a Synolog VM. with TrueNAS itwasn't that hard because both PVE and TrueNAS use KVM as a hypervisor so virtual NICs are both called "vtnetX". But I've read there was a workaround using bonds or something like that so that the NIC name doesn't matter.

Have a look at this: https://www.thomas-krenn.com/en/wiki/OPNsense_HA_Cluster_configuration

 

[DigiDoc]

I thought of that but I would need more hardware and network gear unless you'd running both VMs on the same PVE node. I was thinking to create an NFS share on PVE and have PVE restore from this share. Yes, this choice comes at the cost of some down time but cheaper I guess.

 

[Dunuin]

In case you got a VLAN capable managed switch it isn't that hard. The OPNsense VM on the NAS just shares the same single Gbit NIC with the other services on the NAS, just using other VLANs to isolate it. And as on my NAS is just running the backup OPNsense, that isn't normally actively used, it hasn't to be fast. 1-2GB RAM + 1-2 vCPUs are fine there. Then its not that fast when the primary OPNsense (4 dedicated NICs on a LACP bond, 4GB RAM, 4 vCPUs) goes down but atleast it is working.

 

[DigiDoc]

This sounds interesting, and yesa synology has vSwitch capabilities that I that I need to look into. I have managed switch in use. What about WAN? I have a direct link between my modem and WAN port on the VM?

 

[Dunuin]

Here it looks like this:


My WAN goes from my ISPs router to my managed switch. There I got a dedicated WAN VLAN. Then I use tagged VLAN (sometimes called trunk) so I can send packets tagged with the WAN+DMZ+LAN VLANs over a single NIC. Then on the servers I split those VLAN tagged packets again into untagged packets. But you could also directly let OPNsense handle those VLANs if you set your PVE bridge to "VLAN aware".

 

[DigiDoc]

This makes a lot of sense. I could not wrap my head around the last statement. How would opnsense recognize vlans on trunk port without having a vlan-aware bridge? I'm my current setup, I have WAN comes to NIC0 on the VM, then LAN/VLANs leave tagged through NIC1 to the trunk port on the switch. I am almost certain I don't have vlan aware on those 2 NIC ports but I wanted to get the concept. Thanks again.

 

[Dunuin]

My OPNsense VMs doesn't know anything about the VLANs. They only receive untagged traffic as the VLAN interfaces on the hosts will add/remove VLAN tags. Lets say there is a WAN packet. Its unagged from ISP router to switch. The switch will add a tag. From there it will go over the trunk and the TrueNAS servers NIC to the WAN VLAN interface. the WAN VLAN interface will remove the tag so WAN bridge and the WAN virtio NIC of the VM will only receive untagged traffic.
And the other way round the OPNsense VM will send untagged packets from the virtio NIC over the WAN bridge to WAN VLAN interface. The WAN VLAN interface will tag it and send it tagged over the NIC to the switch. And the switch again will untag that packet as soon as it leaves the switch to the ISP router.
That way you don't need VLAN aware bridges and don't need to configure VLANs on the OPNsense.

But using a VLAN aware bridge, like you described it, should work fine too. I just did it that way because TrueNAS uses FreeBSD and FreeBSD isn't supporting VLAN aware bridges. So that wasn't an option on my TrueNAS server and as both VMs need to use identically named NICs for HA that also means I couldn't use it that way on the PVE server too.

 

[DigiDoc]

Thank you for elaborating. I get it now.

 

[Dunuin]

Forget one thing. I got a dedicated NIC on both servers for cluster comunication. So both servers are directly connected without the swich in between. These NICs I use for the pfsync protocol that needs a very low latency. Benefit of using a dedicated NIC is that big transfers, like sending a backup to the NAS, can't increase the latency. So best practice would be to have a dedicated NIC for pfsync but I guess using the trunk for it should work too. But in case your switch supports Quality of Service (QoS) you might want to create a dedicated VLAN for pfsync and give that VLAN a high priority so pfsync packets are priorized over normal file transfers to/from your NAS.

 

[DigiDoc]

Makes sense. I guess my TL-SG108E will do the job. I may have to reinstall OPNsense all over again as the VM appeared to have consumed all my drive on Proxmox side.... fun learning.