Zstd should give you the best compression of the available compression algorithms.
But if you really want to save space you should use the Proxmox Backup Server (PBS) instead of vzdump. PBS supports incremental backups and deduplication. So backups will be faster as only data that actually changed since the last backup will have to be backuped (even faster when suing snapshot mode instead of stop mode) and nothing as stored more than once. So keeping 20 daily backups of a 100GB VM, which only changes 5GB per day, will use something like 200GB instead of the 2TB that vzdump would need.