Natted Windows VM can ping public IPs but public HTTP(s) is not working

[Andreas Piening]

I have a bridge vmbr1 with the PVE host and some VMs attached to it.
The host uses shorewall to configure iptables. Access from the internal bridge to the public zone is configured by policy. There is masquerading enabled for the public bridge vmbr0 for the internal network.
I can access the internet without limitations from the three windows guests.
All the sudden, accessing external sites via HTTP stops working on one, two or sometimes all three windows VMs at the same time. I can open a CMD prompt and resolve DNS queries, ping public IPs but accessing HTTPS sites with the browser does not work anymore. The request ends with a timeout.
Rebooting the host and VMs doesn't affect the problem, but after a while it magically works for days or even weeks before the issue occurs again.
I know how weird this sounds but that's how it behaves and I have no clue how to dig into that any further.

Any suggestions are greatly appreciated.

 

[Stoiko Ivanov]

hmm - sounds like it's possibly a duplicate MAC or IP on that network?
if you can rule this out - try disabling deny rules in shorewall and see if the problem persists

I hope this helps!

 

[Andreas Piening]

Dear Stoiko,
a duplicate Mac / IP would in fact be a possible cause, but I checked this with

grep -R "virtio=" /etc/pve/qemu-server

and the mac addresses are unique. Same is true for the IP addresses, but this would have been reported by windows anyways I guess.

I should add that the systems are accessed by RDP and there was never any connectivity issue or hiccup while doing that. Even not when the issue accessing external HTTPs sites is just happening, ping to public addresses works with 0 packets lost.
Even more strange: One of the servers does act as a web-server (Tomcat with reverse proxy) and the site is accessible over the local address without issues.

I did a

shorewall clear

that should sort all sorts of blocking. But there are no host specific rules active, there is a zone based accept rule local => net active and all three hosts does share the same network / zone.

 

[Stoiko Ivanov]

a duplicate Mac / IP would in fact be a possible cause, but I checked this with

It could be that the duplicate IP/MAC is on another system (not on the PVE-node), which shares the layer2-network with the PVE-node.
also with the grep you'd catch duplicate MACs but not necessarily duplicate IPs

maybe you can take a closer look with tcpdump whether some packets arrive at the node, while the machines cannot access the outside.
else - check `dmesg` and `journalctl -r` - maybe there's some table or cache which overflows

 

[Andreas Piening]

Oh my god you might be up to something.
I just did a arp lookup on the host and there are two different IP addresses sharing the same HWaddress.
Both are on vmbr1. One is the IP address of an OpenVPN client that dials in and connects to this bridge via tap mode.
The second IP address is the local IP address of the very same system that dials in via OpenVPN and also acts as a gateway (there is a route to the other network targeting the OpenVPN client dial in IP address).

I consider having two IP addresses with the same HWaddress in the ARP table is generally not healthy. Is this true? Or does the fact that they belong to the same system which is an OpenVPN Client and a router at the same time justify for that?

I still can't see why this might create this kind of issue since the internet GW is not affected by this duplicate mac address.
The communication over the OpenVPN tunnel is also working reliably.

 

[Stoiko Ivanov]

I consider having two IP addresses with the same HWaddress in the ARP table is generally not healthy. Is this true?

I would not say that this is bad in general - I have multiple systems, which have more ip's bound to the same interface (=same mac-address) - e.g. for ipv6 - each service gets its own ipv6 address but I don't want to add so many NICs to the box. But in that case it sounds as if you have a MAC-conflict ( 2 separate machines with a the same mac-address) - I would suggest changing one of the virtual macs to a different value.

I hope this helps!