PVE Firewalls and SIP

[arubenstein]

I am having a very strange problem with firewalling a VM, which is a virtual PBX. I've spent a bit of time troubleshooting this and I simply cannot figure out what is happening. I consider myself pretty well-versed in PVE and FW, literally hundreds of VMs across a few clusters, many with complex firewall setups that work just fine. I've made this issue as basic as I can. This is a Linux VM with a VirtIO network card.

If the "Firewall" checkbox is NOT ticked on the network card, everything works fine. Calls in and out all day, no problems. Everything just works. When I tick that box, I immediately have issues sending SIP calls out from the VM to our provider. I've tried:

- Under Options, setting "Firewall" to "NO"
- Under Options, setting "Input Policy" and "Output Policy" to ACCEPT
- Adding rules with "out" and "in" set to "ACCEPT" (all IP)
- Adding rules with "out" and "in" and "Protocol UDP" to ACCEPT (all UDP)

Inbound calls from the SIP provider to the VM work fine, two-way audio is fine as well. It's just outbound calls that are of issue.

I am at a loss. any thoughts?

:tap150i0-IN - [0:0]
:tap150i0-OUT - [0:0]
-A PVEFW-FWBR-IN -m physdev --physdev-out tap150i0 --physdev-is-bridged -j tap150i0-IN
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap150i0 --physdev-is-bridged -j tap150i0-OUT
-A tap150i0-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A tap150i0-IN -j ACCEPT
-A tap150i0-IN -m comment --comment "PVESIG:OFdXzqXcwmyj0szvDL/e5fRT+nI"
-A tap150i0-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A tap150i0-OUT -m mac ! --mac-source 4e:62:60:5f:93:de -j DROP
-A tap150i0-OUT -j MARK --set-xmark 0x0/0x80000000
-A tap150i0-OUT -m limit --limit 1/sec -j NFLOG --nflog-prefix ":150:1:tap150i0-OUT: ACCEPT: "
-A tap150i0-OUT -g PVEFW-SET-ACCEPT-MARK
-A tap150i0-OUT -g PVEFW-SET-ACCEPT-MARK
-A tap150i0-OUT -m comment --comment "PVESIG:DIY5MgeZA0+FJEjKMorU7Qunr1Q"

 

[shanreich]

It might be the same problem as reported in this thread here [1]. I sadly haven't had the time yet to further investigate this, but hopefully I'll be able to look into it more closely soon.


[1] https://forum.proxmox.com/threads/problem-firewall-dropping-packet.153904/

 

[arubenstein]

I've read through that thread. He stated disabling conn-track resolved his problem. I don't know enough about that to know if that would cause a problem for me, but it seems like this would me tcp-established would break.

But it lead me to dig in that regard and I stumbled across this in the PVE docs:

Host related configuration is read from:

/etc/pve/nodes/<nodename>/host.fw

[OPTIONS]

nf_conntrack_helpers: <string> (default =``)
Enable conntrack helpers for specific protocols. Supported protocols: amanda, ftp, irc, netbios-ns, pptp, sane, sip, snmp, tftp

Specifically, "sip"... Thoughts?

Let me know how I can help you shoot this trouble.

 

[KevinS]

If the "Firewall" checkbox is NOT ticked on the network card, everything works fine. Calls in and out all day, no problems. Everything just works. When I tick that box, I immediately have issues sending SIP calls out from the VM to our provider

...

Inbound calls from the SIP provider to the VM work fine, two-way audio is fine as well. It's just outbound calls that are of issue.

In which way does the call not work? Is there no voice, or can the call not be established at all?
Do you use SIP UDP or TCP? Do you use STUN-Servers? Is there a NAT in place?

From personal experience, conntrack issues normally affect inbound calls (as the SIP Port is getting closed after a register). Conntrack SIP helper are pretty much a SIP ALG as far as I can and should also affect inbound calls. Does your SIP provider advise for or against SIP ALG/Helpers?

 

[arubenstein]

To clarify - the vPBX is on PVE. The phone is many miles away on a different IP network (connecting by SIP), and the SIP carrier is very far away on a very distant IP network.

It would seem that the phones connectivity to the vPBX is not of issue. The issue seems to arise when the vPBX tries to connect to the SIP carrier ro send the call into the PSTN, it just claims there is a SIP timeout.

We use UDP, not TCP. No STUN. No NAT.

Our practice has been to not use ALGs at all, often times it creates more problems then fixes.

With all that being said - what I am trying zero in on - why is it not working even if I have an explicit "allow all" rule which should circumvent all firewalling in the first place?

 

[KevinS]

Thanks for the details of your setup, as I've checked the other Thread I want to make sure you're rebooting the VM as you activate the firewall on the VM?

Do you have the firewall enabled at the NIC in the VM configuration? You need to restart the VM after enabling the new firewall to make sure it does not create an additional firewall bridge.

If so, are you currently up to date on the non-subscription repository?
With this information, I'll try to reproduce your setup in our lab to further investigate this behaviour.

 

[arubenstein]

Thanks for the details of your setup, as I've checked the other Thread I want to make sure you're rebooting the VM as you activate the firewall on the VM?

If so, are you currently up to date on the non-subscription repository?
With this information, I'll try to reproduce your setup in our lab to further investigate this behaviour.

- I did not reboot the VM. I can try that.

- I am subscription based but have not updated in a while. We are due. Running 8.1.4.

I am happy to help any way I can

 

[KevinS]

Depending on what else is running on the system, you could take the opportunity and update/reboot the host as well. If not, please let us know if the cold reboot of the VM (shutting down the VM completely and then boot) fixed the issue.

Since you have a subscription, you may also open a support ticket so you can share tcpdumps with us. This way, we can troubleshoot easier and faster.

 

[arubenstein]

I tried enabling the firewall and powering down the VM and powering it back up. Same behavior. Would not pass portions of the SIP conversation until I unchecked the firewall checkbox on the VirtIO card - even with an explicit "allow all" rule. I guess the next thing to do is PVE upgrades. That will take some time.

 

[KevinS]

Thanks for the update. I hope you take up our offer with the support ticket, since we'll probably need a tcpdump to track down the issue. As of now, I can't think of a technical reason why an incoming call (which then sends an SIP invitation to a different network since the phones are at a different location as well) would work, but an outgoing call isn't. In the meantime, I'll try to set something up in our lab and try to reproduce the issue.

 

[arubenstein]

Finally got the cluster updated to 8.3.0 - no change in behavior. Also tried the cold start of the VM with firewalling enabled, no dice.