IPv6 - First-time configuration

[judahnator]

Hello! I am trying to troubleshoot an issue.

The setup:
One host, two guests. The host has a bridge configured with ::1 in a local /48. I am assigning each guest a /56 from this network. See the screenshots-

Host:


Guest:


The problem:
The guests have no real network access over this interface. If I set the IPv6 gateway to the host address I can ping the host, but each guest cannot ping the other.

The solution:
Is hopefully in this thread.
Where do I even start troubleshooting this?

 

[shrdlicka]

Hi,
lets start with the simplest question: Is the firewall enabled on the node/cluster?

 

[judahnator]

Is the firewall enabled on the node/cluster?

It is disabled at the datacenter and VM(s) levels but was enabled on the node itself.
I tried disabling the firewall on the node but no dice, still can't ping the other VM.

 

[shrdlicka]

I'm not 100% on my IPv6 knowledge, but are those two VMs in the same "subnet"? Also are both on the same 'vmbr2' bridge?

 

[judahnator]

are those two VMs in the same "subnet"? Also are both on the same 'vmbr2' bridge?

Yes and yes.

The confusing part is that both guests can ping the host, but neither can ping the other guest.

 

[shrdlicka]

Can you post the /etc/network/interfaces config as well as the conainter configs pct config <container id>?

 

[judahnator]

Here you go!

/etc/network/interfaces

auto lo
iface lo inet loopback

iface enp3s0 inet manual

iface enp2s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.1.199/24
        gateway 192.168.1.1
        bridge-ports enp3s0
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address 10.0.0.1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0
#Internal Network

auto vmbr2
iface vmbr2 inet6 static
        address fd5a:bfc9:4b96::1/48
        bridge-ports none
        bridge-stp off
        bridge-fd 0
#internal ipv6

And for the guests:

root@pve:~# pct config 108
arch: amd64
cores: 1
hostname: ipv6-t1
memory: 512
net0: name=eth0,bridge=vmbr2,firewall=1,gw6=fd5a:bfc9:4b96::1,hwaddr=B2:99:A8:CF:79:CA,ip6=fd5a:bfc9:4b96:0800::1/56,type=veth
ostype: debian
rootfs: local-lvm:vm-108-disk-0,size=8G
swap: 512
unprivileged: 1

and:

root@pve:~# pct config 110
arch: amd64
cores: 1
hostname: ipv6-t2
memory: 512
net0: name=eth0,bridge=vmbr2,firewall=1,gw6=fd5a:bfc9:4b96::1,hwaddr=2A:1F:ED:C5:96:6E,ip6=fd5a:bfc9:4b96:1000::1/56,type=veth
ostype: debian
rootfs: local-lvm:vm-110-disk-0,size=8G
swap: 512
unprivileged: 1

 

[spirit]

you can't access from a /56 network to a /48 ip.

if you want ot use differents /56 network for each subnet/vm, you need multiple /56 ip on your host as gateway too.

here en example, subnetting a /48 (which could be on your main vmbr0 to route upstream), with 2 /56.

auto vmbr0
iface vmbr0 inet static
        address fd65a:bfc9:4b96::/48
        gateway6 ....


auto vmbr2
iface vmbr2 inet6 static
        address fd5a:bfc9:4b96:0800::/56
        bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr3
iface vmbr3 inet6 static
        address fd5a:bfc9:4b96:1000::/56
        bridge-ports none
        bridge-stp off
        bridge-fd 0
[CODE]

(Note that first avaiable ip in a ipv6 subnet is 0 or ::  at the end of the ip )



vm 108:
net0: name=eth0,bridge=vmbr2,firewall=1,gw6=fd5a:bfc9:4b96:0800::,hwaddr=B2:99:A8:CF:79:CA,ip6=fd5a:bfc9:4b96:0800::1/56,type=veth

vm 110
net0: name=eth0,bridge=vmbr3,firewall=1,gw6=fd5a:bfc9:4b96:1000::,hwaddr=2A:1F:ED:C5:96:6E,ip6=fd5a:bfc9:4b96:1000::1/56,type=veth



and enable ipv6 forwarding on host

echo 1 > /proc/sys/net/ipv6/conf/all/forwarding

 

[judahnator]

That makes sense. I plugged that setup in and things are working now.

I'm wondering if it's possible to accomplish this without adding an interface to the host for each guest? Perhaps with a VM instead of a container?
I could set up the /56 networks to be directly routed instead of the /48, the network side of things is flexible.

 

[judahnator]

Following up on my own post-

you can't access from a /56 network to a /48 ip. ... you need multiple ... ip on your host as gateway too

This was the key point here. I misunderstood how the routing was working.

My solution: Use the host bits. This way I don't need a new interface for each VM.