TCP RST Packets dropped by pve-firewall

gslongo

New Member
Apr 20, 2015
21
0
1
Hi All,

We are experiencing a very strange issue or bug. Here is the case :

  • Assume we have a service running on TCP port 12354
  • Clients can communicate with it while running
  • While service is down, clients recieved "Connection timed out" (no answer) even if OS send TCP RST packets :

[root@cluster-dev-1 ~]# tcpdump -ni any tcp port 12345
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
14:28:11.985896 IP 172.16.38.128.41470 > 172.18.0.20.italk: Flags , seq 906491996, win 29200, options [mss 1460,sackOK,TS val 3812309330 ecr 0,nop,wscale 7], length 0
14:28:11.985919 IP 172.18.0.20.italk > 172.16.38.128.41470: Flags [R.], seq 0, ack 906491997, win 0, length 0


However, these RST packets are dropped somewhere in PVE firewall.
On the VM options :

  • Firewall > Options > Firewall = No , Has no effect
  • Firewall > Options > * Policy = ACCEPT , Has no effect (even with NO rule in active for this VM)
  • Hardware > Network Device >firewall=0 , allows packets RST to pass !

Any idea ?

Thank you !

Here is version informations :

~# pveversion -v
proxmox-ve: 5.4-2 (running kernel: 4.15.18-18-pve)
pve-manager: 5.4-11 (running version: 5.4-11/6df3d8d0)
pve-kernel-4.15: 5.4-6
pve-kernel-4.15.18-18-pve: 4.15.18-44
pve-kernel-4.15.18-16-pve: 4.15.18-41
pve-kernel-4.15.18-14-pve: 4.15.18-39
pve-kernel-4.15.18-10-pve: 4.15.18-32
pve-kernel-4.15.18-9-pve: 4.15.18-30
pve-kernel-4.15.18-1-pve: 4.15.18-19
pve-kernel-4.15.17-1-pve: 4.15.17-9
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-12
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-53
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-14
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-37
pve-container: 2.0-39
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2
 

wolfgang

Proxmox Staff Member
Staff member
Oct 1, 2014
4,987
332
83
Hi,

I would update your kernel and test it again.
The kernel you use is quite old and no basis to analyze the problem.
Also, what NIC do you use and how is your network settings (bond, bridge,...)
 

spirit

Well-Known Member
Apr 2, 2010
3,527
156
63
www.odiso.com
if you can reproduce, you can also tcpdump on fwbrX bridge, and vmbrX bridge, and see if packet is dropped somewhere.

(the fwbr bridge is create when you enable firewall on the vm nic)
 

spirit

Well-Known Member
Apr 2, 2010
3,527
156
63
www.odiso.com
also, can you try to change value of sysctl:

nf_conntrack_tcp_be_liberal
"If it's non-zero, we mark only out of window RST segments as INVALID"



(I have seen problem with tcp_be_liberal for a totally different problem, with with some connection marked as invalid, and all packet was dropped by the main conntrack rule)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!