IPv6 problems (maybe from proxmox!)

[m-electronics]

Hallo an die Proxmox-Community,

Ich brauche Schwarmwissen, denn seit ein paar Wochen (fällt es mir zumindest auf) kann ich von Clients hinter einer Router-VM die IP des Routers (`prefix::`) nicht mehr pingen. Und nach Paketanalyse auf der Bridge und auf den VMs selbst, gehen zu dieser IP auch gar keine ICMPv6-Echo-Request-Pakete raus.
Es kommt aber nicht einfach keine Antwort (bzw. das hatte ich jetzt auch schon), sondern stattdessen antwortet der Host von dem der Ping läuft selbst auf den Ping mit seiner IPv6-Addresse auf dem Interface. Und ich dachte erst es liegt an diesem speziellen Router, nämlich MikroTik, aber da es mit VyOS nun auch nicht funktioniert, muss es ja irgendwas auf dem Layer darunter sein.

Wenn jemand eine Idee hat, gerne antworten.
Ich bin mit meinem Latein am Ende.

 

[ggoller]

Hi!
so does the ping succeed or not? how is the interface of your client vm configured?

(note that this is the english forum, the german one is here: https://forum.proxmox.com/forums/proxmox-ve-deutsch-german.20/)

 

[m-electronics]

Hi!
so does the ping succeed or not? how is the interface of your client vm configured?

(note that this is the english forum, the german one is here: https://forum.proxmox.com/forums/proxmox-ve-deutsch-german.20/)

Oh I didn´t see that with the language because in the proxmox datacenter manager forum was also answered in German.
But for your question: The pings succeeds, but the ping answer comes from a wrong IP address. From the IPv6 address host address itself.
The interface is configured normally with a IPv6 address in a /68 subnet (yes, that works) and a default route to the IPv6 address of the router VM.

 

[m-electronics]

No help here anymore? Or did I understand something wrong?

 

[ggoller]

could you post your routes here and the ping command you used?

 

[m-electronics]

could you post your routes here and the ping command you used?

I can do that
1. Ping (DIFFERENT ADDRESS warning!)

2. Routes

And I captured the packets outgoing and coming in on the pfSense machine, and no echo requests are there, but Neighbor Solictitation messages to that aren´t a advertisement message is there as reply, so like the pfSense doesn´t have this address. But this is only with that address `::`, with `::1` I don´t have this problem. And I don´t change anything on my config, only update proxmox and then it must began.

 

[ggoller]

Hi!
could you also send me the tcpdump and the interface configuration of ens18 here and on the router?

 

[m-electronics]

Hi!
could you also send me the tcpdump and the interface configuration of ens18 here and on the router?

I upgraded now the proxmox. At the moment it is working but I don´t know how long that will be so. But I´m now unsure what the originator was.

 

[m-electronics]

The working state was not for a long time
On my other proxmox node (not upgraded, but I didn´t see anything in the updates that is associated with networking) it is the same issue.
And now I have more details: The neighbor solicitation points to a wrong MAC address!
There are more than one device in the same network that is joining the solicited node multicast group address. Reason: ???
The right device also joins it, but doesn´t answer to the neighbor solicitation. The wrong device answers!

Neighbor solicitation:

Neighbor advertisement from wrong address:

(I can´t attach a .pcapng file here)

Interface Configuration:
1. On the client:

2. On the router (pfSense):

 

[ggoller]

Hmm could you try pinging it and then pasting the ip neighbor output here? It seems to me like the router is only reachable over the ...1000::b machine and it proxies the advertisement. Not sure though...

 

[m-electronics]

Hmm could you try pinging it and then pasting the ip neighbor output here? It seems to me like the router is only reachable over the ...1000::b machine and it proxies the advertisement. Not sure though...

I think the second router is joining the solicited node multicast address group for the <prefix>:: although he doesn´t have this address.

 

[m-electronics]

Hmm could you try pinging it and then pasting the ip neighbor output here? It seems to me like the router is only reachable over the ...1000::b machine and it proxies the advertisement. Not sure though...

But what I find very strange is why it is work sometimes and sometimes not.

 

[m-electronics]

Hmm could you try pinging it and then pasting the ip neighbor output here? It seems to me like the router is only reachable over the ...1000::b machine and it proxies the advertisement. Not sure though...

But I have the ip -6 neigh show here:

$ ip -6 neigh show
fd26:ab4d:3844:1000:: dev ens18 lladdr bc:24:11:ce:82:4f router STALE <- Wrong MAC address! (From the second router)
fe80::d807:c4ff:fec1:e068 dev ens18 lladdr da:07:c4:c1:e0:68 router STALE
fe80::be24:11ff:fece:824f dev ens18 lladdr bc:24:11:ce:82:4f router STALE

Here is the right MAC address from the IPv4 Neighbor list:

10.128.8.1 dev ens18 lladdr da:07:c4:c1:e0:68 REACHABLE <- Right MAC address for the IPv4 GW address

 

[ggoller]

Hi! sorry for the delay; I think there is something wrong with your pfSense router because if it correctly joins the multicast group then it should also answer the neighbor solicitations. The MikroTik/Vyos router also answering shouldn't be an issue, because the pfSense advertisement should have the overwrite flag, overriding the MikroTik/Vyos one. Anyway I don't think this a pve issue per se.

 

[m-electronics]

Hi! sorry for the delay; I think there is something wrong with your pfSense router because if it correctly joins the multicast group then it should also answer the neighbor solicitations. The MikroTik/Vyos router also answering shouldn't be an issue, because the pfSense advertisement should have the overwrite flag, overriding the MikroTik/Vyos one. Anyway I don't think this a pve issue per se.

Hey, but I don´t understand why the MikroTik / VyOS is joining a solicited multicast address for an address which they don´t have? And when it isn´t something with proxmox why it began after a proxmox upgrade? (I think so)
But I also think it would be a very strange problem when it comes from proxmox

 

[dMopp]

This might be one of the freeBSD IPv6 issues. Look at the opnsense forum regarding that one :/

 

[ggoller]

Hey, but I don´t understand why the MikroTik / VyOS is joining a solicited multicast address for an address which they don´t have?

AFAIU your setup, the pfsense router is reachable from the MikroTik / VyOS router and so the MikroTik / VyOS router sends a proxied advertisement answer to the soliciation (So basically: "Hey, you can get to the pfSense router if you go through me!"). This is expected.

 

[m-electronics]

AFAIU your setup, the pfsense router is reachable from the MikroTik / VyOS router and so the MikroTik / VyOS router sends a proxied advertisement answer to the soliciation (So basically: "Hey, you can get to the pfSense router if you go through me!"). This is expected.

But it doesn´t work.

 

[m-electronics]

This might be one of the freeBSD IPv6 issues. Look at the opnsense forum regarding that one :/

But I have this issue only when I use the first address of the IPv6 prefix the "::", when I use another like "::1" I don´t have this issue. And when it is an issue from freeBSD why I don´t have it any time? I don´t update any of the pfSense routers with patches or anything.

 

[m-electronics]

AFAIU your setup, the pfsense router is reachable from the MikroTik / VyOS router and so the MikroTik / VyOS router sends a proxied advertisement answer to the soliciation (So basically: "Hey, you can get to the pfSense router if you go through me!"). This is expected.

No, it is also not reachable when I use the first address "::".