migration_unsecure don't work

[groy]

Hi,

2 host link by a cross-over cable, link at 10Gb/s.
migration transfer run at 60MB/sec...

So i have add "migration_unsecure : 1" in datacenter.cfg
reboot my 2 nodes.
And run a new migration.
=> same problem, approximatively 60MB/s

I don't understand why migration run with ssh.
With migration_unsecure it don't (that's the goal of this option, not ?)

I have missed something ?

Thanks,
GR

/etc/pve/datacenter.cfg :

console: html5
keyboard: fr
migration_unsecure: 1

PVEVERSION HOST1 HOST2:

proxmox-ve-2.6.32: 3.3-138 (running kernel: 2.6.32-33-pve)
pve-manager: 3.3-2 (running version: 3.3-2/995e687e)
pve-kernel-2.6.32-33-pve: 2.6.32-138
pve-kernel-2.6.32-29-pve: 2.6.32-126
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-1
pve-cluster: 3.0-15
qemu-server: 3.1-35
pve-firmware: 1.1-3
libpve-common-perl: 3.0-19
libpve-access-control: 3.0-15
libpve-storage-perl: 3.0-23
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8
vzctl: 4.0-1pve6
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 2.1-9
ksm-control-daemon: 1.1-1
glusterfs-client: 3.5.2-1

IPERF :

root@PMXHDV2:~# iperf -c 10.254.254.1
------------------------------------------------------------
Client connecting to 10.254.254.1, TCP port 5001
TCP window size: 19.6 KByte (default)
------------------------------------------------------------
[  3] local 10.254.254.2 port 51196 connected with 10.254.254.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  10.9 GBytes  9.39 Gbits/sec


root@PMXHDV2:~# iperf -c 192.168.1.141
------------------------------------------------------------
Client connecting to 192.168.1.141, TCP port 5001
TCP window size: 19.6 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.142 port 54712 connected with 192.168.1.141 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes   942 Mbits/sec


root@PMXHDV1:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 10.254.254.1 port 5001 connected with 10.254.254.2 port 51196
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  10.9 GBytes  9.39 Gbits/sec
[  5] local 192.168.1.141 port 5001 connected with 192.168.1.142 port 54712
[  5]  0.0-10.0 sec  1.10 GBytes   941 Mbits/sec

MIGRATION :

Oct 27 15:36:19 starting migration of VM 310 to node 'PMXHDV2' (10.254.254.2)
Oct 27 15:36:19 copying disk images
vm-310-disk-1.qcow2


rsync status: 32768   0%    0.00kB/s    0:00:00  
rsync status: 375881728   1%   63.62MB/s    0:08:41  
rsync status: 708345856   2%   63.28MB/s    0:08:39  
rsync status: 1062436864   3%   52.45MB/s    0:10:20  
rsync status: 1392214016   4%   63.07MB/s    0:08:30  
rsync status: 1729134592   5%   48.98MB/s    0:10:50

TOP :

top - 15:37:36 up  1:12,  1 user,  load average: 0.21, 0.09, 0.02
Tasks: 309 total,   3 running, 306 sleeping,   0 stopped,   0 zombie
%Cpu(s):  8.7 us,  1.8 sy,  0.0 ni, 89.1 id,  0.0 wa,  0.0 hi,  0.4 si,  0.0 st
KiB Mem:  65922324 total,  1327984 used, 64594340 free,    13596 buffers
KiB Swap: 16777212 total,        0 used, 16777212 free,   641184 cached


    PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
   8631 root      20   0 73488 6028 2904 R    97  0.0   0:09.36 sshd
   8634 root      20   0 11272  628  384 R    37  0.0   0:03.51 rsync
   8635 root      20   0 23376 1812 1184 R     0  0.0   0:00.02 top

 

[spirit]

Hi,

migration_unsecure is to avoid ssh tunnel from vm memory migration for online migration.

This is not related to disk copy with rsync.

 

[groy]

Hi,
Ok, so it's not possible to deactivate secure rsync copy ?
Another idea to explain my bandwith problem ?

 

[spirit]

>>Ok, so it's not possible to deactivate secure rsync copy ?
Curretly not, it's rsync over ssh. Could be done without ssh, but this need to have a rsync deamon listening on each host.

>Another idea to explain my bandwith problem ?
ssh tunnel are known to be single threaded/use only 1core. So I think this is the bottleneck.

 

[groy]

Ok, thanks.
I will follow this tips topic : http://forum.proxmox.com/threads/17303-migration-slow-disable-encryption

 

[groy]

Better with ssh with Cipher in arcfour128 ..

[COLOR=#000000][FONT=tahoma]rsync status: 28641656832  83%  157.09MB/s    0:00:35  [/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]rsync status: 28970156032  84%  156.96MB/s    0:00:33  [/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]rsync status: 29299310592  85%  156.91MB/s    0:00:31  [/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]rsync status: 29632659456  86%  158.07MB/s    0:00:29  [/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]rsync status: 29961846784  87%  158.08MB/s    0:00:27  [/FONT][/COLOR]

but could do better...
I will try with dedicated rsync daemon and report results.