PMG <> Dovecot Mailbox connectivity issue inside Proxmox PVE 9 test lab - SOLVED

[antfca]

I have spent several days struggling with this....

##TL;DR (shorter):**
In a Proxmox 9.0 lab, my PMG 8.2 VM (10.10.10.14) and Mailbox VM (10.10.10.15, Postfix/Dovecot) can’t talk on port 25. Host ↔ VM works, but VM ↔ VM times out. Likely Proxmox firewall/bridge drop rule. Should I disable VM firewalls or test PMG 9.0 Beta?

## TL;DR (details)
In a **Proxmox 9.0 test lab** (not production), my PMG 8.2.0 VM (10.10.10.14) cannot connect to my Mailbox VM (10.10.10.15, Dovecot 2.4.1 on Debian Trixie).
`nc` from PMG to Mailbox on port 25 times out, even though Postfix is listening on both ends, firewall rules look correct, and ARP entries are fine.
This is **not yet production**: my live system is a separate WHM/cPanel server (CloudLinux + Imunify360) and runs fine. I’m just trying to validate the new stack before migration.
Is this a Proxmox firewall / bridge bug, or am I missing a config detail?
Also: PMG 9.0 Beta just released — is that worth testing here?

## Environment
- **Host**: Proxmox VE 9.0.10 (kernel 6.14.11-2-pve)
- **PMG**: Proxmox Mail Gateway 8.2.0 (kernel 6.8.12-8-pve) — VM 102, IP `10.10.10.14`
- **Mailbox**: Debian Trixie with Dovecot 2.4.1 + Postfix — VM 103, IP `10.10.10.15`
- **Other**: WireGuard VM also on LAN `10.10.10.0/27`

## Symptoms
- From Proxmox host:

nc -vz -w 5 10.10.10.15 25
→ Connection succeeded

• From PMG VM (10.10.10.14):

nc -vz -w 5 10.10.10.15 25
→ Connection timed out

• From Mailbox VM back to PMG:

nc -vz -w 5 10.10.10.14 25
→ Connection timed out

• tcpdump on Proxmox host shows SYN packets leaving one VM and arriving at the other’s tap device, but no SYN-ACK reply ever returns.

### Firewall status
Cluster firewall:
[OPTIONS]
enable: 1

VM103 (Mailbox):
[OPTIONS]
enable: 1
[RULES]
IN ACCEPT
OUT ACCEPT

VM102 (PMG):
[OPTIONS]
enable: 1
[RULES]
#### Allow SSH from LAN
IN ACCEPT -p tcp -source 10.10.10.0/27 -dport 47047
#### Safety-net
IN ACCEPT -p tcp -source 123.123.123.123/32 -dport 47047

No host-level firewall file.
iptables -S shows tap rules with -j DROP at the end, which may be biting us.
A tap102i0-IN -j PVEFW-Drop
A tap102i0-IN -j DROP
A tap103i0-IN -j PVEFW-Drop
A tap103i0-IN -j DROP

and out bound
A tap102i0-OUT -m mac ! --mac-source bc:24:11:15:05:5e -j DROP
A tap103i0-OUT -m mac ! --mac-source bc:24:11:60:1d:e4 -j DROP < that mac is VMAC for the failover IP.

## Versions
#### Proxmox host:
proxmox-ve: 9.0.0 (kernel: 6.14.11-2-pve)
pve-manager: 9.0.10
pve-firewall: 6.0.3

#### PMG VM:
proxmox-mailgateway: 8.2.0
pmg-api: 8.2.0
pmg-gui: 4.1.6

#### "Mailbox" VM:
• Debian Trixie
• Postfix listening on 0.0.0.0:25
• Dovecot 2.4.1
• inet_interfaces = all

## Observations
• Both VMs can ARP each other fine (ip neigh show shows REACHABLE).
• Host ↔ each VM works fine on port 25.
• But VM ↔ VM (PMG ↔ Mailbox) always times out.
• Disabling Proxmox firewall sometimes causes keyboard lag in the GUI until re-enabled, which is suspicious.
• Removing nftables helped simplify, but problem persists.

## Questions
1. Is this a known Proxmox VE 9.0 firewall/bridge bug (tap rules ending with -j DROP)?
2. Should I disable VM firewalls entirely while testing?
3. Any guidance on safe testing of PMG 9.0 Beta in this lab (given we are not yet production)?
• Is upgrading from 8.2 to 9.0 Beta advisable now, or wait for stable?

Thanks in advance — happy to provide more tcpdump/iptables output if useful - hard to get that balance - PLEASE SHOUT IF i MISSED ANY SANITISATION.

p.s. is m.d. supported in here?

 

[antfca]

Solved: VM↔VM SMTP traffic blocked by Proxmox host FORWARD policy
Issue turned out not to be with PMG or the VM firewalls themselves, but with the Proxmox host’s FORWARD chain. Default policy was DROP, and VM↔VM packets on vmbr1 were being discarded before they hit the per-VM firewall chains.

Resolution
Added two explicit host-level FORWARD rules, before the jump to PVEFW-FORWARD < That "before" is crucial.

iptables -I FORWARD 1 -s 10.10.10.14 -d 10.10.10.15 -p tcp --dport 25 -j ACCEPT
iptables -I FORWARD 2 -s 10.10.10.15 -d 10.10.10.14 -p tcp --sport 25 -j ACCEPT

These allow PMG (10.10.10.14) to open TCP/25 to Mailbox (10.10.10.15) and accept the replies. Once in place, SMTP connections worked normally and traces showed full SYN→SYN-ACK→ACK exchange.

Persistence
Rules saved with iptables-persistent and/or enforced with a systemd drop-in to re-assert them after pve-firewall.service. This ensures they survive reboots and Proxmox firewall reloads.
Outcome
Mail path between PMG and Mailbox is now stable and functioning correctly - I might build in an auto re create these on boot update etc
I have to say this feels like a "hack" is there a better way?