Managing ipv6 prefix on Proxmox cluster

[rungekutta]

Hi, I'm running Proxmox 8.0.4 on a two node cluster plus a qdevice (RPi). Works well. My ISP recently enabled ipv6 with a dynamic /56 prefix which also works well. The wan gw acquires the prefix from ISP using WIDE-DHCPv6, and uses dnsmasq as DHCPv4 server as well as SLAAC for ipv6. Dual-stack configuration. All works well and VMs in Proxmox as well as other devices on the network behave as they should and expected.

HOWEVER. Since the /56 prefix is dynamic, I've tried to minimize the number of places I've had to hardcode it in. I've managed to keep it out of firewall rules on the gw altogether using nftables and bitmasking. However I still have it in quite a few places in my Proxmox config, including firewall rules for VM isolation, ipsets for spoof-protection, and static ipv6 addresses for LXC and VMs in order to give the last /64 something that resembles their ipv4 equivalent for clarity.

When my ISP suddenly switched my /56 prefix the other day I manually went through and updated the various config in Proxmox. However this is tedious and error prone. Obviously I could write a script that works itself through all the Promox config files and search/replace old /56 prefix with new. However this seems a bit crude (and could go wrong!).

Is there any best practice here? I'm thinking this must be a common problem for anyone trying to set up an environment in Proxmox with ipv6 and dynamic prefix, which in turn must be quite common as ipv6 becomes more widespread.


...?

 

[rungekutta]

PS. note to mods - apologies if posted in the wrong forum, feel free to move to networking if that’s where it belongs..!

 

[BobhWasatch]

It is pretty much like IPv4. You use a private subnet for local traffic. But since each device also gets a delegated global address you don't need to do NAT. Each device gets at least a delegated address, a local address, and a link-local (fe80 prefix).

I use OpenWRT on a cheap Celeron mini-pc as my gateway router (you can also use a VM if you prefer). My ISP gateway is set in "DMZ+" mode which is the closest AT&T offers to a bridge mode. It is basically just a pass-through to my router.

OpenWRT has a setting under Network > Interfaces > Global Network Options where you can enter a private prefix or it will generate one automatically if you don't enter any. It will give out addresses in this range to clients on the LAN side via DHCPv6 and SLAAC along with the delegated range from your ISP. In OpenWRT's DNS setup I assign names to the static 4/6 IP's and let DHCP/DHCPv6 handle the dynamic names.

I have not used pfsense but I believe it has similar features and may be more popular than OpenWRT for running in a VM. I've not seen an ISP-provided gateway that does this stuff, but they may exist.

ETA: It is like how IPv4 would work if we didn't have a shortage of addresses

 

[rungekutta]

Hi @BobhWasatch and thanks for your answer! You misunderstand my question though. Maybe I wasn't clear.

I have no problems at the gateway side of things. All devices on my network have a link-local address (generated by the device itself) in the fe80::/10 block, a ULA (unique local address) in the fd00::/8 block generated with SLAAC from a prefix provided by the gw, and one or several global IPv6 addresses generated with SLAAC from a prefix provided by the gw which in turn gets its global /56 prefix from the ISP upstream. I use neither OpenWRT or Pfsense for this but rather a vanilla Debian configured from standard packages (nftables, dnsmasq, unbound, wide-dhcpv6-client, etc). All this works very well and is relatively uncomplicated.

Some of my services are exposed to the internet. In such case, the gw simply allows access to global addresses on my network from incoming traffic on the wan. No need for NAT or DNAT as you say. And I've managed to set up those nftables rules in a way which doesn't care about the /56 prefix (the first 56 bits of the 2001:... global addresses) which is good because it means I don't have to reconfigure anything when my ISP switches prefix.

However I haven't managed to achieve the same in Proxmox. There are several places where I've had to hardcode the dynamic global /56 prefix in config. Including but not limited to:

1) Ipset/ipfilter to protect from IP spoofing (https://pve.proxmox.com/pve-docs/chapter-pve-firewall.html#pve_firewall_ipfilter_section). This requires me to list all acceptable IP-addresses for the VM in Proxmox config including its global address which includes the dynamic prefix from the ISP.

2) Proxmox VM isolation: in my IPv4 equivalent, I isolate all VMs on DMZ from one another by blocking 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 and only explicitly allowing what's required (for email, file shares etc). That way I can achieve a level of isolation equivalent to each VM running in its own dedicated DMZ. Taking the same concept to IPv6, I can easily block fd00::/8 and fe80::/10 in the same way BUT I also want to block VMs from accessing each other via their global 2001:.. address directly, unless explicitly allowed. I can do this easily by blocking cross-talk om my global /56 prefix - BUT then another example of the dynamic prefix hardcoded in Proxmox config.

All this leads to the last 2 paragraphs in my opening post; when my ISP suddenly switched my /56 prefix the other day I manually went through and updated the various config in Proxmox, which was tedious and error prone. I could write a script that works itself through all the Proxmox config files and search/replace old /56 prefix with new but this seems a bit crude and hacky.

So wondering if there is any best practice here? Or am I simply one of the first ones trying to do these things in Proxmox on top of IPv6 and a dynamic global prefix...?

 

[BobhWasatch]

Yeah, I misunderstood your question. Hosting services on a dynamic IP is a problem. Perhaps you could get a static delegation from your ISP but I suspect most of them want you to have a business account for that. Short of that I don't have any better ideas than what you're already doing.

 

[rungekutta]

Thanks for your suggestions. Any recommendation where I would start looking in order to get started with such a setup? Haven’t used Ansible or Puppet before.

 

[Willi_H]

Hi, because now the dark time of the year has started again, I have also, as a new goal, planned to implement IPv6 in my Homelab. I also only get a dynamic IPv6 prefix from the provider. Therefore I would like to join "rungekutta" and kindly ask for a possible guide or starting point for new research to implement a dynamic setting with Puppet or Ansible.