[HOWTO] proxmox 3.4->4.2 upgrade, with qemu live migration

[spirit]

Hi, here a howto to do proxmox 3.4 -> 4.2 cluster upgrade, with qemu live migration without interruption

1)Upgrade a first node to proxmox 4.2 and recreate cluster
------------------------------------------------------------
Have an empty node,
then upgrade it to proxmox 4.2, folowing the current wiki


# apt-get update && apt-get dist-upgrade
# apt-get remove proxmox-ve-2.6.32 pve-manager corosync-pve openais-pve redhat-cluster-pve pve-cluster pve-firmware
# sed -i 's/wheezy/jessie/g' /etc/apt/sources.list
# sed -i 's/wheezy/jessie/g' /etc/apt/sources.list.d/pve-enterprise.list
# apt-get update
# apt-get install pve-kernel-4.4.8-1-pve pve-firmware
# apt-get dist-upgrade

reboot

# apt-get install proxmox-ve
# apt-get remove pve-kernel-2.6.32-41-pve

# pvecm create <clustername>


2) upgrade second node
----------------------
# apt-get update && apt-get dist-upgrade
# apt-get remove proxmox-ve-2.6.32 pve-manager corosync-pve openais-pve redhat-cluster-pve pve-cluster pve-firmware
# sed -i 's/wheezy/jessie/g' /etc/apt/sources.list
# sed -i 's/wheezy/jessie/g' /etc/apt/sources.list.d/pve-enterprise.list
# apt-get update
# apt-get install pve-kernel-4.4.8-1-pve pve-firmware
# apt-get dist-upgrade


---> here no reboot

now the tricky part

# cp /bin/systemctl /
# echo "" > /bin/systemctl 

# apt-get install proxmox-ve

if /etc/pve is not mounted:
# /usr/bin/pmxcfs -l

add node to cluster

# pvecm add ipofnode1 -force

--->waiting for quorum message : ctrl-c

close old corosync and delete old config
# killall -9 corosync
# /etc/init.d/pve-cluster stop
# rm /var/lib/pve-cluster/config.db*

start new corosync and pve-cluster

# corosync
# /etc/init.d/pve-cluster start

verify than you can write in /etc/pve/ and that's is correctly replicate on other proxmox4 nodes
#touch /etc/pve/test.txt
#rm /etc/pve/test.txt

migrate vms (do it for each vmid)
-----------------------------------
verify that you have in /etc/pve/datacenter.cfg
migration_unsecure: 1

# qm migrate <vmid> <target_proxmox4_server> -online

(migrate must do done with cli, because pvestatd can't start without systemd, so gui is not working)

#cp /systemctl /bin/



# reboot node



3) do the same thing for next node(s)

4) when all nodes are migrated, remove

# rm /etc/pve/cluster.conf

 

[mir]

By empty node you mean an old node where all VMs are migrated away?

 

[spirit]

By empty node you mean an old node where all VMs are migrated away?

yes.
(because we need to reboot it)

 

[mir]

Given the new openvswitch version in pve-4.2 would you recommend switching from linux bridge to openvswitch?

 

[spirit]

Given the new openvswitch version in pve-4.2 would you recommend switching from linux bridge to openvswitch?

I'm still currently on linux bridge but with vlan filtering enabled.
For now, I don't see any performance difference.


I'm currently working to implemente dpdk on openvswitch 2.5 + qemu vhost-user, it should improve a lot performance for small packets (10gits/s 64bytes ). something like from 1mpps to 12mpps. For router or firewall vms, under synflood attack, it's really a big improvement