I had the same issue. With about 192 GB RAM, I had to keep 5-10% free. That totaled to 10-20 GB of RAM, sitting around unused.
In the end, I found the culprit: kexec hand over (KHO). This is a relativley new feature. There were discussions about it on the proxmox kernel mailing list.
The...