PVE: edit mac filter rule / modify ebtables rule

[colch]

Hello dear proxmox forum!

TLDR: I want to allow particular MAC through mac filter rule. How I could edit ebtables rule for particular VM, so it will fit ergonomically into pve itself?

I tried searching proxmox forum before creating new thread. Well.. I tried searching internet for this. But looks like I'm alone with this problem

Feel free to correct me at any point, as I'm frontend developer for life, and playing with proxmox / things at my homelab just for fun. BTW, proxmox is awesome, I really love it.

So, my setup is:

* L0 vm is ProxMox itself
* L1 vm is Alpine with docker installed
* L2 is python3 webserver container

For networking,

* ProxMox: vmbr0 to eno1 interface. vmbr0 is VLAN aware; and vmbr.20 is VLAN for all vm's at proxmox
* Alpine: eth1 is tied to vmbr0, with VLAN tag 20
* As for python3-webserver, I want to use macvlan docker network

Macvlan network on docker is tied to vm's eth1 (e.g. parent=eth1), with same subnet, etc.

The problem is, when I try to `docker exec python3 sh` into that container (as root), and try to ping/dig/tracert anything, there is no ... no connection, I guess. It's just timeouting all the things.

Sure, I was testing all things with firewall turned off, just to make sure it's not firewall.

And, turns out, there is mac filtering feature at alpine options. When I turn off this feature, container starts to work normally. I could reach it via my usual router network, just like it's my regular machine. And, of course, ping/dig/tracert inside of container works normally in this case.

I guess, this is called Mac spoofing. Docker container is using different mac, that is assigned to VM, and inputs/outputs network packets with it.

Anyway, I figured out (I don't remember how, may be some link in pve docs), that for mac filtering, ebtables is used.

I started to digging into it, and, indeed, for my VM I see this rule:

```
Bridge chain: veth101i0-OUT, entries: 2, policy: ACCEPT
-s ! bc:24:11:98:44:1d -j DROP
-j ACCEPT
```

I don't want to turn off mac filtering, because I want to keep this layer of security. But I want to edit this ebtables rule, so it will allow me to use mac 02:42:0a:14:00:02, for example.

So, finally, question is : how I could edit ebtables rule for VM, so it will fit ergonomically into pve itself? I mean, I want to do it so things inside of pve will not break: speaking of removing VM, adding new VM, even booting pve and so on.

 

[spirit]

I'll be hard to add this this yourself.

ask this feature on bugzilla.proxmox.com (the new nftables implementation is currently in developpement, so it's a good time to ask for it)

 

[colch]

I'll be hard to add this this yourself.

ask this feature on bugzilla.proxmox.com (the new nftables implementation is currently in developpement, so it's a good time to ask for it)

yes, good idea! created a bug for this https://bugzilla.proxmox.com/show_bug.cgi?id=5416

 

[colch]

Aside from that, I found the place how it's exactly used https://github.com/proxmox/pve-fire...c65c9c0d18f95c8f222/src/PVE/Firewall.pm#L2436

    if ($direction eq 'OUT') {
    if (defined($macaddr) && !(defined($options->{macfilter}) && $options->{macfilter} == 0)) {
        ruleset_addrule($ruleset, $chain, "-m mac ! --mac-source $macaddr", "-j DROP");
    }

so it's a iptables.

inded, I see this rule:

exists veth102i0-OUT (tFVuPWZge8V88M/Z7KRwzMPIab0)
    -A veth102i0-OUT -p udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
    -A veth102i0-OUT -m mac ! --mac-source BC:24:11:C8:66:F8 -j DROP
    -A veth102i0-OUT -j MARK --set-mark 0x00000000/0x80000000
    -A veth102i0-OUT -j GROUP-in-mgnt-OUT
    -A veth102i0-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN

anyway, I can't modify those enough to be able to fit into proxmox iptables rules.

one thought it so remove this DROP rule and add 2 ACCEPT rules + DROP OTHERS

iptables -I veth102i0-OUT 2 -m mac --mac-source 02:42:0a:14:00:02 -j ACCEPT
iptables -I veth102i0-OUT 2 -m mac --mac-source bc:24:11:e4:8a:bf -j ACCEPT
iptables -D veth102i0-OUT -m mac ! --mac-source bc:24:11:e4:8a:bf -j DROP
iptables -A veth102i0-OUT -j DROP

but this will not fit into proxmox iptables rules well

I guess I have to disable mac filtering feature...

---------

I tried to somewhat simulate my desired behaviour. I just DROP anything egress, and allow only specific IP's in specific interfaces

10.20.0.8 is IP of this machine (docker host)
10.20.0.15 is IP of a docker container with macvlan

as this is mac spoof, I will use static arp on my mikrotik

root@ASUS-MINIPC:~# iptables -L veth102i0-OUT --line-numbers -n -v
Chain veth102i0-OUT (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 PVEFW-SET-ACCEPT-MARK  17   --  *      *       0.0.0.0/0            0.0.0.0/0           [goto]  udp spt:68 dpt:67
2        9   746 MARK       0    --  *      *       0.0.0.0/0            0.0.0.0/0            MARK and 0x7fffffff
3        9   746 GROUP-in-mgnt-OUT  0    --  *      *       0.0.0.0/0            0.0.0.0/0
4        0     0 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x80000000/0x80000000
5        9   746 GROUP-in-portainer-OUT  0    --  *      *       0.0.0.0/0            0.0.0.0/0
6        0     0 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x80000000/0x80000000
7        9   746 GROUP-in-opn-router-OUT  0    --  *      *       0.0.0.0/0            0.0.0.0/0
8        0     0 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x80000000/0x80000000
9        9   746 GROUP-out-step-ca-OUT  0    --  *      *       0.0.0.0/0            0.0.0.0/0
10       0     0 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x80000000/0x80000000
11       9   746 GROUP-out-no-lan-OUT  0    --  *      *       0.0.0.0/0            0.0.0.0/0
12       1    74 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x80000000/0x80000000
13       0     0 PVEFW-SET-ACCEPT-MARK  0    --  *      *       10.20.0.8            0.0.0.0/0           [goto]
14       2   168 PVEFW-SET-ACCEPT-MARK  0    --  *      *       10.20.0.15           0.0.0.0/0           [goto]
15       0     0 PVEFW-Drop  0    --  *      *       0.0.0.0/0            0.0.0.0/0
16       0     0 NFLOG      0    --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 1/sec burst 5 nflog-prefix ":102:7:veth102i0-OUT: policy DROP: "
17       0     0 DROP       0    --  *      *       0.0.0.0/0            0.0.0.0/0
18       0     0            0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* PVESIG:Muyah9zzEY+Xxnbi1nwkhA2+H3w */