[HOW-TO] Separate migration network - dirty fix

MeyrNL

New Member
Feb 23, 2015
20
0
1
Hi all,

On the forum I saw topics about questions how to let Proxmox Cluster use a different network for migration traffic.
Currently for all my clusters I made a code change in the QemuServer.pm to change the listening IP (unfortunately hard-coded) of the migration task.

Currently I have 5 interfaces per hypervisor:
  • NIC1 and NIC2 (2 x Gigabit Ethernet) are configured in bonding only for public network traffic - no ip address configured - connect to two different gigabit switches.
  • NIC3 (1 x Gigabit Ethernet) is configured as management traffic. Only for internal traffic - ip address: 10.0.10.XX - connected to gigabit switch.
  • NIC4 and NIC5 (2 x 10Gigabit Ethernet) are configured in bonding only for storage traffic of DRBD, use of NFS etc - ip address: 10.0.7.XX - connected to a 10G switch.

I know this is not the neatest way. Please note that if you update the Proxmox to a newer version, the changes will be lost.

HOW-TO:

  • [*=1]First check the IP of network you would like to use for the migration network. In my case I use the storage network also for my live migration network (ip: 10.0.7.48 as example).
    [*=1]Open the following file in your favourite file editor: /usr/share/perl5/PVE/QemuServer.pm
    [*=1]Search in the file for the variable: $migrate_uri
    The result will show the following: $migrate_uri = "tcp:${localip}:${migrate_port}";
    [*=1]By replacing the "${localip}" with the IP you would like to use, Proxmox will be forced to listen on that ip address everytime he receives a migration request.
    The result will be as following after the replacement (in my example with use of ip 10.0.7.48): $migrate_uri = "tcp:10.0.7.48:${migrate_port}";
    [*=1]Save the file and make sure it is saved by running the command:
    root@hypervisor48:~# cat /usr/share/perl5/PVE/QemuServer.pm | grep migrate_uri
    my $migrate_uri;
    $migrate_uri = "tcp:10.0.7.48:${migrate_port}";
    push @$cmd, '-incoming', $migrate_uri;
    print "migration listens on $migrate_uri\n" if $migrate_uri;

    [*=1]Restart the pvedaemon on the server by typing the following:
    root@hypervisor48:~# service pvedaemon restart
    Restarting PVE Daemon: pvedaemon.

    [*=1]Do the above steps on every node which are listening on that same network.
    [*=1]Test the changes by doing a live migration of a VM. You will see if the changes have worked in the sentence: starting online/live migration on 10.0.7.48:pORT

If you have any questions, please do not hesitate to contact me.
 
Last edited:

sdutremble

Active Member
Sep 29, 2011
85
0
26
Would it make sense to define a variable "migrateip"?

We could set it at a similar location as for localip (for example set migrateip to localip initially so the code works the same as it is now). We could even make it part of the GUI at a later date as a drop-down to select the available networks defined for the cluster.

This is a minor logical change of the code that would allow much flexibility.

Any thought/comment?

Serge
 

sdutremble

Active Member
Sep 29, 2011
85
0
26
I think that would be great. However, I suggest a drop-down with only the configured network.

Serge
 

swartz

Member
Oct 30, 2014
80
3
6
Yes, I would also like to see a way to specify migration (or maintenance) network.

This would offload main interface(s) from "maintenance" traffic.
 
Mar 19, 2016
9
0
21
Hi, sorry for bumping on this quite old post.

I have 2x 1Gb on vmbr0 and 2x 10Gb on a failover-bond - this where i`d like to have migration-traffic.

I had a quick look at QemuServer.pm in latest release.
Obviously the Hack described by the op is still necessary to get migration-traffic on a separate interface.

For now i can live with that, but on the long run it`s quite annoying to edit QemuServer.pm after every update

So are there any plans to replace

$migrate_uri = "tcp:${localip}:${migrate_port}";

in Line 4526 by something that picks the IP from a config-file, with fallback to the localip ?
Guess it wouldn't matter if it`s accessible on the GUI or not.
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
6,734
1,176
164
please file a feature request ("enhancement") at bugzilla.proxmox.com .
 

gkovacs

Well-Known Member
Dec 22, 2008
509
48
48
Budapest, Hungary
Hi all,

On the forum I saw topics about questions how to let Proxmox Cluster use a different network for migration traffic.
Currently for all my clusters I made a code change in the QemuServer.pm to change the listening IP (unfortunately hard-coded) of the migration task..

Thank you for this. Two questions:
- is this supposed to work on Proxmox 3.x or only 4.x?
- is it possible to adapt this for OpenVZ migration (rsync) somehow?
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!