MAC Address abuse report?!

[openaspace]

Hello.
I have received a MAC abuse notification ...where my server is using n°3 mac address that aren't allowed for my account!

I have verified all MAC address configured on the virtual machines and proxmox host are all correct no one of the 3 mentioned MAC address are used in my configuration.

But looking in the proxmox firewall log .. I see DROP incoming connections to one of the tap device...where the destination IP of the dropped connection it's not configured on my server and the mac address correspond to 1 of 3 mentioned in the abuse notification.

200 6 tap200i0-IN 04/Sep/2021:22:11:30 +0200 policy DROP: IN=fwbr200i0 OUT=fwbr200i0 PHYSIN=fwln200i0 PHYSOUT=tap200i0 MAC=e8:06:88:ca:33:ff SRC=REMOTE-IP DST=IP-NOT-OWNED-BY-ME LEN=44 TOS=0x00 PREC=0x00 TTL=40 ID=25740 PROTO=TCP SPT=34435 DPT=40001 SEQ=643497095 ACK=0 WINDOW=1024 SYN

Also tcpdump confirm incoming (only incoming..) connections addressed to the 3 MAC for IP's who are not in my ip pool.

I use Hetzner server, could be because I use bridged net instead of routed net?!
What's going on?!

 

[spirit]

mmm, ask to hertzner, maybe they are flooding bridged BUM traffic from another customer to you. (maybe a bug in their fdb bridge table or filtering, I really don't known).

 

[dendi]

Hello, I have the same issue with hetzner from yesterday and currently no solution.
I see (only) incoming traffic to vmbr0 adressed to some mac addresses. I have no VMs on vmbr0 as I use a separate bridge with private addresses with a NAT rule on vmbr0.
I wrote to hetzner that I can't find those mac addresses on my system and got this reply: "and your server reply because?".
I'm not sure my server replies, I only see that traffic with tcpdump... any idea?

 

[spirit]

just do a "tcpdump -e" to see mac address.

It's quite possible that hertzner send you traffic, even if you don't have the mac address : if they don't known the mac address on their switches, it's possible that they are flooding packets on their network until this mac respond.
Now, that's strange that layer2 is not splitted across customers with vlan,vxlan,...

 

[dendi]

Hello Spirit, this is a sample output of tcpdump -en

14:38:12.606071 MAC_OF_HETZNER_SW > FAKE_MAC, ethertype IPv4 (0x0800), length 66: UNKNOWN_NET_IP.12202 > IP_IN_MY_CLASS.80: Flags [S], seq 3272021817, win 65535, options [mss 1460,nop,nop,sackOK,nop,wscale 12], length 0
14:38:13.018601 MAC_OF_HETZNER_SW > FAKE_MAC, ethertype IPv4 (0x0800), length 60: UNKNOWN_NET_IP.25565 > IP_IN_MY_CLASS.80: Flags [SE], seq 0, win 65535, length 0

That's really strange, my server seems to hit some traffic of others hetzner servers but I don't find that "fake" mac addresses with ifconfig and arp -e

 

[openaspace]

14:38:12.606071 MAC_OF_HETZNER_SW > FAKE_MAC, ethertype IPv4 (0x0800), length 66: UNKNOWN_NET_IP.12202 > IP_IN_MY_CLASS.80: Flags [S], seq 3272021817, win 65535, options [mss 1460,nop,nop,sackOK,nop,wscale 12], length 0

I also see incoming traffic to the unallowed mac address reported by the datacenter

tcpdump ether host d4:3d:7e:d8:c0:dc
15:09:52.060311 IP REMOTE.IP.41580 > SERVER.IP.47409: Flags , seq 2338244102, win 1024, length 0

where the DST server IP it's not owned by me!

 

[openaspace]

ifconfig and arp -e

use: tcpdump ether host MAC_ADDRESS

also if you make a simple search ctrl-f in the proxmox server firewall log window with the abused mac you will find it in a DROP firewall line.

I'm migrating some vps to AWS..for live streaming events.. ..(pfffffff)

 

[openaspace]

Now, that's strange that layer2 is not splitted across customers with vlan,vxlan,...

yeah.. "no comment!"

 

[openaspace]

What I observe in my configuration is that I have not applied this network configuration in my server,
where each public ip is declared manually on the bridge:

https://www.indivar.com/blog/how-to...-dedicated-server/#host-network-configuration

 

[spirit]

I think you should send tcpdump trace to hertzner support, they really need to that on their network, you can't do nothing about this.

 

[dendi]

I think you should send tcpdump trace to hertzner support, they really need to that on their network, you can't do nothing about this.

Already done and got this reply: "and your server reply because?".
I never reply but after about one hour, I received another automatic email saying: "We have accepted your statement on the issue. The ticket has now been closed."
I'm in their hands...

 

[spirit]

Already done and got this reply: "and your server reply because?".
I never reply but after about one hour, I received another automatic email saying: "We have accepted your statement on the issue. The ticket has now been closed."
I'm in their hands...

so ": "and your server reply because?" --> just tell them than your server don't reply with tcpdump traces, but that it's them that send crap on your server. (and if they see replies, just ask them to give some proof,traces,...)

 

[openaspace]

I really can't realize because my server receive traffic for different ip for not present MAC Address...
I have verified my bridged configuration as hetzner advice: https://community.hetzner.com/tutor...ure-proxmox_ve#step-2---network-configuration
and it's ok...

the only possibility is to switch to routed mode?!

 

[dendi]

I really can't realize because my server receive traffic for different ip for not present MAC Address...
I have verified my bridged configuration as hetzner advice: https://community.hetzner.com/tutor...ure-proxmox_ve#step-2---network-configuration
and it's ok...

the only possibility is to switch to routed mode?!

I think you should read this: https://forum.proxmox.com/threads/proxmox-claiming-mac-address.52601/

 

[Semmo]

I got the same mail from Hetzner. And the reported MAC addresses belong to some of the fwbr devices (Firwall Bridge from Proxmox). That's weird.. that shouldn't happen?! But they closed the case before I reported back

 

[spirit]

I got the same mail from Hetzner. And the reported MAC addresses belong to some of the fwbr devices (Firwall Bridge from Proxmox). That's weird.. that shouldn't happen?! But they closed the case before I reported back

proxmox version ?

 

[Semmo]

proxmox version ?

proxmox-ve: 6.4-1 (running kernel: 5.4.143-1-pve)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-helper: 6.4-8
pve-kernel-5.4: 6.4-7
pve-kernel-5.3: 6.1-6
pve-kernel-5.0: 6.0-11
pve-kernel-5.4.143-1-pve: 5.4.143-1
pve-kernel-5.4.140-1-pve: 5.4.140-1
pve-kernel-5.4.128-1-pve: 5.4.128-2
pve-kernel-5.4.124-1-pve: 5.4.124-2
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.0.21-5-pve: 5.0.21-10
pve-kernel-5.0.15-1-pve: 5.0.15-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ifupdown2: not correctly installed
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.22-pve1~bpo10+1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.1.0-1
libpve-access-control: 6.4-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-4
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.13-2
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.6-1
pve-cluster: 6.4-1
pve-container: 3.3-6
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.3-2
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.6-pve1~bpo10+1

 

[spirit]

proxmox-ve: 6.4-1 (running kernel: 5.4.143-1-pve)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-helper: 6.4-8
pve-kernel-5.4: 6.4-7
pve-kernel-5.3: 6.1-6
pve-kernel-5.0: 6.0-11
pve-kernel-5.4.143-1-pve: 5.4.143-1
pve-kernel-5.4.140-1-pve: 5.4.140-1
pve-kernel-5.4.128-1-pve: 5.4.128-2
pve-kernel-5.4.124-1-pve: 5.4.124-2
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.0.21-5-pve: 5.0.21-10
pve-kernel-5.0.15-1-pve: 5.0.15-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ifupdown2: not correctly installed
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.22-pve1~bpo10+1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.1.0-1
libpve-access-control: 6.4-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-4
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.13-2
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.6-1
pve-cluster: 6.4-1
pve-container: 3.3-6
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.3-2
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.6-pve1~bpo10+1

they are now "bugs" on proxmox 6.x (fixed in proxmox 7) with some packets sent with the wrong mac

if you don't use proxmox firewall, disable the firewall on vm nic, it should fix it.

if you use proxmox firewall:

- never use REJECT in rules, or default inbound rules on vms firewall
- add a rules in each vm "direction in - drop - tcp - port 43"
- add in "/etc/sysctl.d/pve.conf" : "net.ipv4.igmp_link_local_mcast_reports = 0" (and reboot)

 

[openaspace]

- never use REJECT in rules, or default inbound rules on vms firewall
- add a rules in each vm "direction in - drop - tcp - port 43"
- add in "/etc/sysctl.d/pve.conf" : "net.ipv4.igmp_link_local_mcast_reports = 0" (and reboot)

therefore this completely solve this "star wars saga" ?!

 

[spirit]

therefore this completely solve this "star wars saga" ?!

I don't have see any user report after theses changes. (sysctl && drop tcp43 are fixed in pve7)