Change host address PVE uses for cluster ssh

[phemmer]

When doing things like VM transfers between hosts in a cluster, PVE uses ssh (https://pve.proxmox.com/wiki/Cluster_Manager#_role_of_ssh_in_proxmox_ve_clusters). In my setup, my cluster members have multiple network interfaces, an "admin" interface, and a "data" interface. The admin interface is what matches the host's hostname, and what DNS resolves to, but it's only a 1Gb link. The data interface is a 20Gb link, and what should be used for most activity, including VM transfers between cluster members.

The problem is that PVE is doing these transfers over the admin interface, while I want it to use the data interface. Originally I was thinking I could add an entry in /etc/ssh/ssh_config to override the address used when connecting by name. However when I looked at the ssh being invoked, it's connecting by IP address:

/usr/bin/ssh -e none -o BatchMode=yes -o HostKeyAlias=ded4461 root@1.2.3.4 -o ExitOnForwardFailure=yes -L /run/qemu-server/100.migrate:/run/qemu-server/100.migrate -L /run/qemu-server/100.migrate:/run/qemu-server/100.migrate /usr/sbin/qm mtunnel

While I could technically still do the ssh_config override on IP address, that just smells like a really bad idea. I would prefer if I could either get PVE to use hostnames for ssh, or change the address it uses. However I'm unsure how to accomplish either of these.
I've tried swapping the ring0 and ring1 addresses in corosync, but that had no effect.

 

[dietmar]

The problem is that PVE is doing these transfers over the admin interface, while I want it to use the data interface.

You may want to set the "migration" option in the datacenter config (see "man datacenter.cfg"). You can specify the network there.

 

[phemmer]

Thanks, that does indeed look like the right way to address it, however it doesn't work in my situation. The problem is that the setting takes a subnet to search for a source address in, but my cluster members aren't in the same subnet. And because /etc/pve is replicated across all cluster members, I can't set a different value for each host.

 

[phemmer]

After digging through the source code, it doesn't look like there's any way to accomplish this. Without datacenter.cfg, it uses hostname resolution, thus the current behavior. And when using datacenter.cfg, all hosts must be on the same subnet.

As such I've filed a feature request on the bug tracker to address this limitation: https://bugzilla.proxmox.com/show_bug.cgi?id=3273