[SOLVED] IPv6 via Tunnelbroker

[Mark-A]

I have a connection via tunnelbroker.net and tried to get it running on PVE 7.1-7 and had no success.
I then loaded 8.0-2 on a small machine, and still no success.

Went to he.net support, they tried it and it appears to connect, the tunnel does come up, verified by support at he.net and by ping.
I have used https://tools.keycdn.com/ipv6-ping and get a ping success from the inside ip on the VM that is running the tunnel-gateway that is running under PVE. When i shut down that VM the ping fails, thus verifying both the positive and negative possibilities.
BUT....
cant get any further than that VM, all other VM's and LXC's wont connect via IPv6.

Doing searches found its probably the vmbr0 needed an ip6 address assigned to it so that it creates a fe80 link-local address, and it did not fix it.
also adding net.ipv6.conf.vmbr0.accept_ra=2 into /etc/symctl.d/00-sysctl.conf or /etc/sysctl.d/local.conf so i did both, and still does not work.
also using a separate bridge with no ipv4 address worked, i created a second bridge vmbr666 and still does not work.
also uncommenting forwarding= in /etc/symctl.d/00-sysctl.conf and still does not work.

I have also discovered pings internal to PVE between machines that some work and some do not.

gateway VM ::2

test web LXC ::4
test web LXC ::5
test web VM ::6

from ::4, ::5 and ::6 i can ping each other and ::2

from ::2 i can NOT ping ::4 ::5 or ::6
but i can ping ::1 that is on the other side of the tunnel and itself ::2 and another address that is an IPV6 dns server.

SO..... what am i doing wrong?

----------------------------

here is some additional info I know someone will ask for:

Package versions
proxmox-ve: 7.1-1 (running kernel: 5.13.19-2-pve)
pve-manager: 7.1-7 (running version: 7.1-7/df5740ad)
pve-kernel-helper: 7.1-6
pve-kernel-5.13: 7.1-5
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph: 16.2.7
ceph-fuse: 16.2.13-pve1
corosync: 3.1.5-
pve2criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-4
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.1.2-1
proxmox-backup-file-restore: 2.1.2-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-4
pve-cluster: 7.1-2
pve-container: 4.1-2
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-3
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3

---------------------------

cat /etc/sysctl.d/99-sysctl.conf
#
# /etc/sysctl.conf - Configuration file for setting system variables
# See /etc/sysctl.d/ for additional system variables.
# See sysctl.conf (5) for information.
#

#kernel.domainname = example.com

# Uncomment the following to stop low-level messages on console
#kernel.printk = 3 4 1 3

###################################################################
# Functions previously found in netbase
#

# Uncomment the next two lines to enable Spoof protection (reverse-path filter)
# Turn on Source Address Verification in all interfaces to
# prevent some spoofing attacks
#net.ipv4.conf.default.rp_filter=1
#net.ipv4.conf.all.rp_filter=1

# Uncomment the next line to enable TCP/IP SYN cookies
# See http://lwn.net/Articles/277146/
# Note: This may impact IPv6 TCP sessions too
#net.ipv4.tcp_syncookies=1

# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1

# Uncomment the next line to enable packet forwarding for IPv6
# Enabling this option disables Stateless Address Autoconfiguration
# based on Router Advertisements for this host
net.ipv6.conf.all.forwarding=1


###################################################################
# Additional settings - these settings can improve the network
# security of the host and prevent against some network attacks
# including spoofing attacks and man in the middle attacks through
# redirection. Some network environments, however, require that these
# settings are disabled so review and enable them as needed.
#
# Do not accept ICMP redirects (prevent MITM attacks)
#net.ipv4.conf.all.accept_redirects = 0
#net.ipv6.conf.all.accept_redirects = 0
# _or_
# Accept ICMP redirects only for gateways listed in our default
# gateway list (enabled by default)
# net.ipv4.conf.all.secure_redirects = 1
#
# Do not send ICMP redirects (we are not a router)
#net.ipv4.conf.all.send_redirects = 0
#
# Do not accept IP source route packets (we are not a router)
#net.ipv4.conf.all.accept_source_route = 0
#net.ipv6.conf.all.accept_source_route = 0
#
# Log Martian Packets
#net.ipv4.conf.all.log_martians = 1
#

###################################################################
# Magic system request Key
# 0=disable, 1=enable all, >1 bitmask of sysrq functions
# See https://www.kernel.org/doc/html/latest/admin-guide/sysrq.html
# for what other values do
#kernel.sysrq=438
net.ipv6.conf.vmbr0.accept_ra=2
net.ipv6.conf.vmbr666.accept_ra=2

---------------------------


cat /etc/sysctl.d/local.conf
net.ipv6.conf.vmbr0.accept_ra=2
net.ipv6.conf.vmbr666.accept_ra=2


----------------------------

/etc/network# cat interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.0.138/24
gateway 192.168.0.1
bridge-ports eth0
bridge-stp off
bridge-fd 0
iface eth0 inet6 auto

iface vmbr0 inet6 static
address fd00::192:168:0:138/64
gateway 2001:470:1c:223::2

auto vmbr666
iface vmbr666 inet6 static
address fd00::b666:192:168:0:138/64
bridge-ports none
bridge-stp off
bridge-fd 0


--------------------------
VM ::2

mark@he-ipv6-gateway:~$ ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group defaul t qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP gro up default qlen 1000
link/ether 76:73:8a:27:53:c9 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.250/20 brd 192.168.15.255 scope global ens18
valid_lft forever preferred_lft forever
inet6 2001:470:1c:223::2/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::7473:8aff:fe27:53c9/64 scope link
valid_lft forever preferred_lft forever
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
link/sit 0.0.0.0 brd 0.0.0.0
4: he-ipv6@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UN KNOWN group default qlen 1000
link/sit 192.168.0.250 peer 216.66.38.58
inet6 2001:470:1c:223::2/64 scope global
valid_lft forever preferred_lft forever



----------------------
LXC ::4

root@HE-test4-web:~# ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: net2@if106: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether f2:51:94:5f:d6:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.1.203/20 brd 192.168.15.255 scope global net2
valid_lft forever preferred_lft forever
inet6 2001:470:1d:223::4/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::f051:94ff:fe5f:d6e6/64 scope link
valid_lft forever preferred_lft forever


------------------------------------------
LXC ::5

root@HE-test5-web:~# ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: net2@if110: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 6a:80:82:25:c7:c4 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.1.205/20 brd 192.168.15.255 scope global net2
valid_lft forever preferred_lft forever
inet6 2001:470:1c:223::5/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::6880:82ff:fe25:c7c4/64 scope link
valid_lft forever preferred_lft forever


-----------------

 

[Mark-A]

additional Info

the following 2 items were supplied by tunnelbroker.net
NOTE the warning on the second

---------------------------
Paste the following into /etc/network/interfaces:

auto he-ipv6
iface he-ipv6 inet6 v4tunnel
address 2001:###:##:223::2
netmask 64
endpoint 216.###.###.###
local 173.###.###.###
ttl 255
gateway 2001:###:##:223::1
--------------------------
Paste the following into a file in /etc/netplan/ named as anything ending in .yaml, such as 99-he-tunnel.yaml to ensure it applied after any other configurations. Running 'sudo netplan apply' should add in these changes. ***NOTE***: This example is new, and not heavily tested. Please note any issues to ipv6@he.net

network:
version: 2
tunnels:
he-ipv6:
mode: sit
remote: 216.###.###.###
local: 173.###.###.###
addresses:
- "2001:###:##:223::2/64"
routes:
- to: default
via: "2001:###:##:223::1"
----------------------------

 

[Mark-A]

I finally got this working.



In proxmox you need to do 1 change, add an IPv6 address to the bridge, then reboot, I used fd00::200 because the IPv4 address ends in a .200.



I used ubuntu for the gateway/tunnel.

What tunnelbroker.net gave for debian/ubuntu did not work.

I used the Linux netplan 0.103+

delete off the 2 first lines then added it to the bottom of the netplan.

My internal address is 192.168.0.250 and the IPv4 gateway is 192.168.0.1

they gave 2 addresses for the tunnel my side 2001:#:1c:#::2 and their side 2001:#:1c:#::1 and i find that you can not use any other addresses in that /64.

they route 2001:#:1d:#::

After trying many things i added 2001:#:1d:#::1 to the address and it began to work.

I then clicked to add a /48, and it began working when i added 2001:#:#::1/48 to the address



network:

ethernets:

ens18:

addresses:

- 192.168.0.250/20

- 2001:#:1c:#::2/64

- 2001:#:1d:#::1/64

- 2001:#:#::1/48

gateway4: 192.168.0.1

nameservers:

addresses:

- 8.8.8.8

- 2001:470:20::2

search: []

#

version: 2

#

tunnels:

he-ipv6:

mode: sit

remote: 216.#.#.#

local: 192.168.0.250

addresses:

- "2001:#:1c:#::2/64"

routes:

- to: default

via: "2001:#:1c:#::1"

#



This is what i found works in one of my test vm’s netplan:



network:

ethernets:

ens18:

addresses:

- 192.168.1.206/20

- 2001:#:1d:#::6/64

gateway4: 192.168.0.1

gateway6: 2001:#:1d:#::1

nameservers:

addresses:

- 8.8.8.8

- 2001:470:20::2

search: []

version: 2



For an LXC add both IPv6 address and gateway Then add the IPv6 dns 2001:470:20::2 to the dns.





FYI

2 tools i found extremely helpful

IPv6 Ping and traceroute https://tools.keycdn.com/ipv6-ping

IPv6 proxy to view your page http://www.ipv6proxy.net/

 

[nikatjef]

Paste the following into /etc/network/interfaces:

auto he-ipv6
iface he-ipv6 inet6 v4tunnel
address 2001:###:##:223::2
netmask 64
endpoint 216.###.###.###
local 173.###.###.###
ttl 255
gateway 2001:###:##:223::1
--------------------------

Not sure about pre Debian 12, but with Debian 12, this is changed to;

auto he-ipv6
iface he-ipv6 inet6 tunnel
        mode sit
        address 2001:###:##:223::2
        netmask 64
        endpoint 216.###.###.###
        local 173.###.###.###
        ttl 255
        gateway 2001:###:##:223::1

Debian 12 uses ifup2 which requires that you set the tunnel mode instead of using a specific tunnel.