Ah, answering my own question from the documentation.
Configuration Files
All firewall related configuration is stored on the proxmox cluster file system. So those files are automatically distributed to all cluster nodes, and the pve-firewall service updates the underlying iptables rules automatically on changes.
(https://pve.proxmox.com/wiki/Firewall#_firewall_rules)