Hey - how to permanently disable IPv6 - it is possible to disable it but after reboot proxmox enables it. Also removing this module is not pentameter as kernel update rebuilt it.
IIRC, there are some IPv6-related settings in /etc/sysctl.conf, try to disable it there (something like "ipv6.disable", try to set all those to "1")...