Hi,
we use Proxmox since a few weeks to host some VMs with public IPs. The hoster firewall is very limited, so we decided to use the Proxmox firewall at datacenter level to filter all inbound traffic. We enabled nftables as we need to filter the forward chain.
After all firewalls at datacenter, host and VM level are enabled and a SSH rule is inplace, we failed to connect the VM with SSH. We used nftrace (https://github.com/aojea/nftrace) for troubleshooting.
We see the connection setup between the external ssh cliend and the internal VM. The initial package with the SYN flag passes the firewall successfully, but the SYN-ACK answer will be dropped in cluster-forward chain.
Traced outbound traffic:
Why is the package dropped in the cluster-forward rule?
Is it possible that connection tracking is not active for the forward chain? As I only found the connection tracking (
What's wrong with my firewall configuration and how can I fix it?
Thank you for your support,
Carsten
Our setup:
PVE: 192.168.1.156
VM network: 172.1.1.33/29
VM: 172.1.1.36
SSH client: 172.12.1.126
we use Proxmox since a few weeks to host some VMs with public IPs. The hoster firewall is very limited, so we decided to use the Proxmox firewall at datacenter level to filter all inbound traffic. We enabled nftables as we need to filter the forward chain.
After all firewalls at datacenter, host and VM level are enabled and a SSH rule is inplace, we failed to connect the VM with SSH. We used nftrace (https://github.com/aojea/nftrace) for troubleshooting.
We see the connection setup between the external ssh cliend and the internal VM. The initial package with the SYN flag passes the firewall successfully, but the SYN-ACK answer will be dropped in cluster-forward chain.
Traced outbound traffic:
Code:
root@pve ~ # ./nftrace add ip daddr 193.101.57.126 tcp sport 22
root@pve ~ # ./nftrace monitor
trace id b7f99d04 inet nftrace-table nftrace-chain packet: iif "vmbr1" ether saddr bc:24:11:39:29:68 ether daddr d6:87:13:59:b9:22 ip saddr 172.1.1.36 ip daddr 172.12.1.126 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 0 ip protocol tcp ip length 52 tcp sport 22 tcp dport 10400 tcp flags == 0x12 tcp window 64240
trace id b7f99d04 inet nftrace-table nftrace-chain rule ip daddr 172.12.1.126 tcp sport 22 meta nftrace set 1 (verdict continue)
trace id b7f99d04 inet nftrace-table nftrace-chain policy accept
trace id b7f99d04 inet proxmox-firewall forward packet: iif "vmbr1" oif "enp5s0" ether saddr bc:24:11:39:29:68 ether daddr d6:87:13:59:b9:22 ip saddr 172.1.1.36 ip daddr 172.12.1.126 ip dscp cs0 ip ecn not-ect ip ttl 63 ip id 0 ip protocol tcp ip length 52 tcp sport 22 tcp dport 10400 tcp flags == 0x12 tcp window 64240
trace id b7f99d04 inet proxmox-firewall forward rule jump host-forward (verdict jump host-forward)
trace id b7f99d04 inet proxmox-firewall forward rule jump cluster-forward (verdict jump cluster-forward)
trace id b7f99d04 inet proxmox-firewall cluster-forward rule drop (verdict drop)
Code:
root@pve ~ # conntrack -L
[...]
tcp 6 52 SYN_RECV src=172.12.1.126 dst=172.1.1.36 sport=10400 dport=22 src=172.1.1.36 dst=172.12.1.126 sport=22 dport=10400 mark=0 use=1
[...]Why is the package dropped in the cluster-forward rule?
Is it possible that connection tracking is not active for the forward chain? As I only found the connection tracking (
ct state vmap { invalid : jump invalid-conntrack, established : accept, related : accept }) in default-in and default-out chains, but not in forward chain nor any childs.What's wrong with my firewall configuration and how can I fix it?
Thank you for your support,
Carsten
Our setup:
PVE: 192.168.1.156
VM network: 172.1.1.33/29
VM: 172.1.1.36
SSH client: 172.12.1.126
Code:
root@pve ~ # cat /etc/pve/firewall/cluster.fw
[OPTIONS]
policy_forward: DROP
enable: 1
[IPSET srv-jump]
172.1.1.36 # jump
[IPSET proxmoxhost]
192.168.1.156 # host
[RULES]
FORWARD SSH(ACCEPT) -dest +dc/srv-jump -log nolog
FORWARD Ping(ACCEPT) -dest +dc/srv-jump -log nolog
IN Ping(ACCEPT) -i vmbr1 -dest +dc/proxmoxhost -log nolog # Interne Pings zwischen VMs und Host
GROUP proxmox -i enp5s0
[group proxmox] # Verwaltung
IN ACCEPT -dest +dc/proxmoxhost -p tcp -dport 8006 -log nolog
IN Ping(ACCEPT) -dest +dc/proxmoxhost -log nolog
IN SSH(ACCEPT) -dest +dc/proxmoxhost -log nolog
Code:
root@pve ~ # cat /etc/network/interfaces
source /etc/network/interfaces.d/*
auto lo
iface lo inet loopback
auto enp5s0
iface enp5s0 inet static
address 192.168.1.156/26
gateway 192.168.1.129
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up echo 1 > /proc/sys/net/ipv4/conf/enp5s0/proxy_arp
auto vmbr1
iface vmbr1 inet static
address 172.1.1.33/29
bridge-ports none
bridge-stp off
bridge-fd 0
Code:
root@pve ~ # qm config 102
boot: order=scsi0;net0;ide2
cores: 2
cpu: x86-64-v2-AES
ide2: local:iso/debian-12.9.0-amd64-DVD-1.iso,media=cdrom,size=3887968K
memory: 4096
meta: creation-qemu=9.0.2,ctime=1741353068
name: srv-jump.example.com
net0: virtio=BC:24:11:39:29:68,bridge=vmbr1,firewall=1
numa: 0
ostype: l26
scsi0: local:102/vm-102-disk-0.qcow2,iothread=1,size=16G
scsi1: local:102/vm-102-disk-1.qcow2,iothread=1,size=10G
scsihw: virtio-scsi-single
smbios1: uuid=760734d8-6075-4fe3-b5dd-6425e1faf9cb
sockets: 1
vmgenid: 0833084c-ae2a-4dbf-ad4b-8a3b446afe6f
Last edited: