Virtual machine crashes after HA migration

O

otisou

Active Member
Apr 24, 2019
2
0
41
42
Hello everyone:
I add a host to my proxmox cluster, Virtual machine migration to other hosts will not be a problem , Migration to a newly added host will cause a crash.

syslog

Sep 15 13:48:56 pveh0 QEMU[143766]: kvm: warning: TSC frequency mismatch between VM (3599999 kHz) and host (2499998 kHz), and TSC scaling unavailable
Sep 15 13:48:56 pveh0 QEMU[143766]: kvm: warning: TSC frequency mismatch between VM (3599999 kHz) and host (2499998 kHz), and TSC scaling unavailable
Sep 15 13:48:56 pveh0 QEMU[143766]: kvm: warning: TSC frequency mismatch between VM (3599999 kHz) and host (2499998 kHz), and TSC scaling unavailable
Sep 15 13:48:56 pveh0 QEMU[143766]: kvm: warning: TSC frequency mismatch between VM (3599999 kHz) and host (2499998 kHz), and TSC scaling unavailable
Sep 15 13:48:56 pveh0 QEMU[143766]: kvm: warning: TSC frequency mismatch between VM (3599999 kHz) and host (2499998 kHz), and TSC scaling unavailable
Sep 15 13:48:56 pveh0 QEMU[143766]: kvm: warning: TSC frequency mismatch between VM (3599999 kHz) and host (2499998 kHz), and TSC scaling unavailable
Sep 15 13:48:56 pveh0 QEMU[143766]: kvm: warning: TSC frequency mismatch between VM (3599999 kHz) and host (2499998 kHz), and TSC scaling unavailable
Sep 15 13:48:56 pveh0 QEMU[143766]: kvm: warning: TSC frequency mismatch between VM (3599999 kHz) and host (2499998 kHz), and TSC scaling unavailable
Sep 15 13:48:56 pveh0 systemd[1]: session-100.scope: Succeeded.
Sep 15 13:48:56 pveh0 systemd[1]: session-100.scope: Consumed 34.685s CPU time.
Sep 15 13:48:57 pveh0 systemd[1]: Started Session 101 of user root.
Sep 15 13:48:57 pveh0 systemd[1]: session-101.scope: Succeeded.
Sep 15 13:48:58 pveh0 systemd[1]: Started Session 102 of user root.
Sep 15 13:49:00 pveh0 systemd[1]: session-102.scope: Succeeded.
Sep 15 13:49:00 pveh0 systemd[1]: session-102.scope: Consumed 1.358s CPU time.
Sep 15 13:49:00 pveh0 pmxcfs[1461]: [status] notice: received log
Sep 15 13:49:04 pveh0 pvedaemon[1787]: VM 110 qmp command failed - VM 110 qmp command 'guest-ping' failed - got timeout
Sep 15 13:49:27 pveh0 pvedaemon[1789]: VM 110 qmp command failed - VM 110 qmp command 'guest-ping' failed - got timeout
Sep 15 13:49:49 pveh0 pvedaemon[1787]: VM 110 qmp command failed - VM 110 qmp command 'guest-ping' failed - got timeout
Sep 15 13:50:09 pveh0 systemd[1]: Started Check_MK LibreNMS Agent Service (10.88.20.90:38232).
Sep 15 13:50:11 pveh0 pvedaemon[1788]: VM 110 qmp command failed - VM 110 qmp command 'guest-ping' failed - got timeout

Thanks
 
Are the hosts the same Hardware/CPU?
What Hardware?
What Guest-Config?
 
fabian said:
https://bugzilla.proxmox.com/show_bug.cgi?id=4073
https://bugzilla.proxmox.com/show_bug.cgi?id=4218
Click to expand...
I am still having issues with live migrating from a Dell PE640 to a Dell PE630. The reverse migration works just fine 630 > 640. So if this still an issue and how can it be worked around? I don't see any resolution in the bug reports. I did check the time on all servers in the cluster and they are in sync.

Both servers have the latest Dell BIOS and firmware version.

Code: 
Dell 640
proxmox-ve: 7.4-1 (running kernel: 5.15.104-1-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-5.15: 7.4-1
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.104-1-pve: 5.15.104-2
pve-kernel-5.15.64-1-pve: 5.15.64-1
pve-kernel-5.15.60-2-pve: 5.15.60-2
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph-fuse: 15.2.15-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-4
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.5
libpve-storage-perl: 7.4-2
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
openvswitch-switch: 2.15.0+ds1-2+deb11u4
proxmox-backup-client: 2.4.1-1
proxmox-backup-file-restore: 2.4.1-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.1-1
proxmox-widget-toolkit: 3.6.5
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-2
pve-firewall: 4.3-1
pve-firmware: 3.6-4
pve-ha-manager: 3.6.0
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.9-pve1

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          48
On-line CPU(s) list:             0-47
Thread(s) per core:              2
Core(s) per socket:              12
Socket(s):                       2
NUMA node(s):                    2
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           85
Model name:                      Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GH
                                 z
Stepping:                        4
CPU MHz:                         2600.000
BogoMIPS:                        5200.00
Virtualization:                  VT-x
L1d cache:                       768 KiB
L1i cache:                       768 KiB
L2 cache:                        24 MiB
L3 cache:                        38.5 MiB
NUMA node0 CPU(s):               0,2,4,6,8,10,12,14,16,18,20,22,24,26,28
                                 ,30,32,34,36,38,40,42,44,46
NUMA node1 CPU(s):               1,3,5,7,9,11,13,15,17,19,21,23,25,27,29
                                 ,31,33,35,37,39,41,43,45,47
Vulnerability Itlb multihit:     KVM: Mitigation: Split huge pages
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditio
                                 nal cache flushes, SMT vulnerable
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Mmio stale data:   Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Vulnerability Retbleed:          Mitigation; IBRS
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass di
                                 sabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers an
                                 d __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; IBRS, IBPB conditional, STI
                                 BP conditional, RSB filling, PBRSB-eIBR
                                 S Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic
                                  sep mtrr pge mca cmov pat pse36 clflus
                                 h dts acpi mmx fxsr sse sse2 ss ht tm p
                                 be syscall nx pdpe1gb rdtscp lm constan
                                 t_tsc art arch_perfmon pebs bts rep_goo
                                 d nopl xtopology nonstop_tsc cpuid aper
                                 fmperf pni pclmulqdq dtes64 monitor ds_
                                 cpl vmx smx est tm2 ssse3 sdbg fma cx16
                                  xtpr pdcm pcid dca sse4_1 sse4_2 x2api
                                 c movbe popcnt tsc_deadline_timer aes x
                                 save avx f16c rdrand lahf_lm abm 3dnowp
                                 refetch cpuid_fault epb cat_l3 cdp_l3 i
                                 nvpcid_single pti intel_ppin ssbd mba i
                                 brs ibpb stibp tpr_shadow vnmi flexprio
                                 rity ept vpid ept_ad fsgsbase tsc_adjus
                                 t bmi1 hle avx2 smep bmi2 erms invpcid
                                 rtm cqm mpx rdt_a avx512f avx512dq rdse
                                 ed adx smap clflushopt clwb intel_pt av
                                 x512cd avx512bw avx512vl xsaveopt xsave
                                 c xgetbv1 xsaves cqm_llc cqm_occup_llc
                                 cqm_mbm_total cqm_mbm_local dtherm ida
                                 arat pln pts pku ospke md_clear flush_l
                                 1d arch_capabilities

Code: 
Dell 630
proxmox-ve: 7.4-1 (running kernel: 5.15.104-1-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-5.15: 7.4-1
pve-kernel-5.15.104-1-pve: 5.15.104-2
pve-kernel-5.15.102-1-pve: 5.15.102-1
ceph-fuse: 15.2.17-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-4
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.5
libpve-storage-perl: 7.4-2
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
openvswitch-switch: 2.15.0+ds1-2+deb11u4
proxmox-backup-client: 2.4.1-1
proxmox-backup-file-restore: 2.4.1-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.6.5
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-2
pve-firewall: 4.3-1
pve-firmware: 3.6-4
pve-ha-manager: 3.6.0
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.9-pve1

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          48
On-line CPU(s) list:             0-47
Thread(s) per core:              2
Core(s) per socket:              12
Socket(s):                       2
NUMA node(s):                    2
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           79
Model name:                      Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20G
                                 Hz
Stepping:                        1
CPU MHz:                         1200.667
CPU max MHz:                     2900.0000
CPU min MHz:                     1200.0000
BogoMIPS:                        4400.20
Virtualization:                  VT-x
L1d cache:                       768 KiB
L1i cache:                       768 KiB
L2 cache:                        6 MiB
L3 cache:                        60 MiB
NUMA node0 CPU(s):               0,2,4,6,8,10,12,14,16,18,20,22,24,26,28
                                 ,30,32,34,36,38,40,42,44,46
NUMA node1 CPU(s):               1,3,5,7,9,11,13,15,17,19,21,23,25,27,29
                                 ,31,33,35,37,39,41,43,45,47
Vulnerability Itlb multihit:     KVM: Mitigation: Split huge pages
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditio
                                 nal cache flushes, SMT vulnerable
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Mmio stale data:   Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass di
                                 sabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers an
                                 d __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, IBPB conditiona
                                 l, IBRS_FW, STIBP conditional, RSB fill
                                 ing, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic
                                  sep mtrr pge mca cmov pat pse36 clflus
                                 h dts acpi mmx fxsr sse sse2 ss ht tm p
                                 be syscall nx pdpe1gb rdtscp lm constan
                                 t_tsc arch_perfmon pebs bts rep_good no
                                 pl xtopology nonstop_tsc cpuid aperfmpe
                                 rf pni pclmulqdq dtes64 monitor ds_cpl
                                 vmx smx est tm2 ssse3 sdbg fma cx16 xtp
                                 r pdcm pcid dca sse4_1 sse4_2 x2apic mo
                                 vbe popcnt tsc_deadline_timer aes xsave
                                  avx f16c rdrand lahf_lm abm 3dnowprefe
                                 tch cpuid_fault epb cat_l3 cdp_l3 invpc
                                 id_single pti ssbd ibrs ibpb stibp tpr_
                                 shadow vnmi flexpriority ept vpid ept_a
                                 d fsgsbase tsc_adjust bmi1 hle avx2 sme
                                 p bmi2 erms invpcid rtm cqm rdt_a rdsee
                                 d adx smap intel_pt xsaveopt cqm_llc cq
                                 m_occup_llc cqm_mbm_total cqm_mbm_local
                                  dtherm ida arat pln pts md_clear flush
                                 _l1d

Thanks,
Eric
 
ejmerkel said:
I am still having issues with live migrating from a Dell PE640 to a Dell PE630. The reverse migration works just fine 630 > 640. So if this still an issue and how can it be worked around? I don't see any resolution in the bug reports. I did check the time on all servers in the cluster and they are in sync.

Both servers have the latest Dell BIOS and firmware version.

Code: 
Dell 640
proxmox-ve: 7.4-1 (running kernel: 5.15.104-1-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-5.15: 7.4-1
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.104-1-pve: 5.15.104-2
pve-kernel-5.15.64-1-pve: 5.15.64-1
pve-kernel-5.15.60-2-pve: 5.15.60-2
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph-fuse: 15.2.15-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-4
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.5
libpve-storage-perl: 7.4-2
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
openvswitch-switch: 2.15.0+ds1-2+deb11u4
proxmox-backup-client: 2.4.1-1
proxmox-backup-file-restore: 2.4.1-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.1-1
proxmox-widget-toolkit: 3.6.5
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-2
pve-firewall: 4.3-1
pve-firmware: 3.6-4
pve-ha-manager: 3.6.0
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.9-pve1

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          48
On-line CPU(s) list:             0-47
Thread(s) per core:              2
Core(s) per socket:              12
Socket(s):                       2
NUMA node(s):                    2
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           85
Model name:                      Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GH
                                 z
Stepping:                        4
CPU MHz:                         2600.000
BogoMIPS:                        5200.00
Virtualization:                  VT-x
L1d cache:                       768 KiB
L1i cache:                       768 KiB
L2 cache:                        24 MiB
L3 cache:                        38.5 MiB
NUMA node0 CPU(s):               0,2,4,6,8,10,12,14,16,18,20,22,24,26,28
                                 ,30,32,34,36,38,40,42,44,46
NUMA node1 CPU(s):               1,3,5,7,9,11,13,15,17,19,21,23,25,27,29
                                 ,31,33,35,37,39,41,43,45,47
Vulnerability Itlb multihit:     KVM: Mitigation: Split huge pages
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditio
                                 nal cache flushes, SMT vulnerable
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Mmio stale data:   Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Vulnerability Retbleed:          Mitigation; IBRS
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass di
                                 sabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers an
                                 d __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; IBRS, IBPB conditional, STI
                                 BP conditional, RSB filling, PBRSB-eIBR
                                 S Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic
                                  sep mtrr pge mca cmov pat pse36 clflus
                                 h dts acpi mmx fxsr sse sse2 ss ht tm p
                                 be syscall nx pdpe1gb rdtscp lm constan
                                 t_tsc art arch_perfmon pebs bts rep_goo
                                 d nopl xtopology nonstop_tsc cpuid aper
                                 fmperf pni pclmulqdq dtes64 monitor ds_
                                 cpl vmx smx est tm2 ssse3 sdbg fma cx16
                                  xtpr pdcm pcid dca sse4_1 sse4_2 x2api
                                 c movbe popcnt tsc_deadline_timer aes x
                                 save avx f16c rdrand lahf_lm abm 3dnowp
                                 refetch cpuid_fault epb cat_l3 cdp_l3 i
                                 nvpcid_single pti intel_ppin ssbd mba i
                                 brs ibpb stibp tpr_shadow vnmi flexprio
                                 rity ept vpid ept_ad fsgsbase tsc_adjus
                                 t bmi1 hle avx2 smep bmi2 erms invpcid
                                 rtm cqm mpx rdt_a avx512f avx512dq rdse
                                 ed adx smap clflushopt clwb intel_pt av
                                 x512cd avx512bw avx512vl xsaveopt xsave
                                 c xgetbv1 xsaves cqm_llc cqm_occup_llc
                                 cqm_mbm_total cqm_mbm_local dtherm ida
                                 arat pln pts pku ospke md_clear flush_l
                                 1d arch_capabilities

Code: 
Dell 630
proxmox-ve: 7.4-1 (running kernel: 5.15.104-1-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-5.15: 7.4-1
pve-kernel-5.15.104-1-pve: 5.15.104-2
pve-kernel-5.15.102-1-pve: 5.15.102-1
ceph-fuse: 15.2.17-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-4
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.5
libpve-storage-perl: 7.4-2
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
openvswitch-switch: 2.15.0+ds1-2+deb11u4
proxmox-backup-client: 2.4.1-1
proxmox-backup-file-restore: 2.4.1-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.6.5
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-2
pve-firewall: 4.3-1
pve-firmware: 3.6-4
pve-ha-manager: 3.6.0
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.9-pve1

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          48
On-line CPU(s) list:             0-47
Thread(s) per core:              2
Core(s) per socket:              12
Socket(s):                       2
NUMA node(s):                    2
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           79
Model name:                      Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20G
                                 Hz
Stepping:                        1
CPU MHz:                         1200.667
CPU max MHz:                     2900.0000
CPU min MHz:                     1200.0000
BogoMIPS:                        4400.20
Virtualization:                  VT-x
L1d cache:                       768 KiB
L1i cache:                       768 KiB
L2 cache:                        6 MiB
L3 cache:                        60 MiB
NUMA node0 CPU(s):               0,2,4,6,8,10,12,14,16,18,20,22,24,26,28
                                 ,30,32,34,36,38,40,42,44,46
NUMA node1 CPU(s):               1,3,5,7,9,11,13,15,17,19,21,23,25,27,29
                                 ,31,33,35,37,39,41,43,45,47
Vulnerability Itlb multihit:     KVM: Mitigation: Split huge pages
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditio
                                 nal cache flushes, SMT vulnerable
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Mmio stale data:   Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass di
                                 sabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers an
                                 d __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, IBPB conditiona
                                 l, IBRS_FW, STIBP conditional, RSB fill
                                 ing, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; Clear CPU buffers; SMT vuln
                                 erable
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic
                                  sep mtrr pge mca cmov pat pse36 clflus
                                 h dts acpi mmx fxsr sse sse2 ss ht tm p
                                 be syscall nx pdpe1gb rdtscp lm constan
                                 t_tsc arch_perfmon pebs bts rep_good no
                                 pl xtopology nonstop_tsc cpuid aperfmpe
                                 rf pni pclmulqdq dtes64 monitor ds_cpl
                                 vmx smx est tm2 ssse3 sdbg fma cx16 xtp
                                 r pdcm pcid dca sse4_1 sse4_2 x2apic mo
                                 vbe popcnt tsc_deadline_timer aes xsave
                                  avx f16c rdrand lahf_lm abm 3dnowprefe
                                 tch cpuid_fault epb cat_l3 cdp_l3 invpc
                                 id_single pti ssbd ibrs ibpb stibp tpr_
                                 shadow vnmi flexpriority ept vpid ept_a
                                 d fsgsbase tsc_adjust bmi1 hle avx2 sme
                                 p bmi2 erms invpcid rtm cqm rdt_a rdsee
                                 d adx smap intel_pt xsaveopt cqm_llc cq
                                 m_occup_llc cqm_mbm_total cqm_mbm_local
                                  dtherm ida arat pln pts md_clear flush
                                 _l1d

Thanks,
Eric
Click to expand...

The solution is to use a more recent kernel than the default 5.15 one on the source and target node(s): [1] [2] [3].
This would currently be the 6.2 opt-in kernel: [4], since all others in-between are EOL.

[1] https://bugzilla.proxmox.com/show_bug.cgi?id=4073#c60
[2] https://forum.proxmox.com/threads/p...on-linux-freeze-on-windows.109645/post-511185 (Again: 5.19 is EOL!)
[3] https://forum.proxmox.com/threads/p...on-linux-freeze-on-windows.109645/post-511210 (Again: 5.19 is EOL!)
[4] https://forum.proxmox.com/threads/opt-in-linux-6-2-kernel-for-proxmox-ve-7-x-available.124189
 
Last edited:
  • Like
Reactions: ejmerkel
@ejmerkel have you tried setting the CPU type of the VM to the older architecture of the two servers? Should be Broadwell in your case.
 
bbgeek17 said:
I'd be curious on what the official PVE recommendation is for Enterprise customers.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Click to expand...
We don't discourage the use of the newer opt-in kernels. It is a valid choice in some setups, for example to get better support for newer hardware, getting some features that are only available in newer kernels, or in some situations it can lead to better stability.

edit: fixed sentence structure ;)
 
Last edited:
aaron said:
@ejmerkel have you tried setting the CPU type of the VM to the older architecture of the two servers? Should be Broadwell in your case.
Click to expand...
I tried setting the CPU type to Broadwell, stopping/starting the VM and then attempting to live migrate it. Unfortunately, the VM still locked up after the migration was complete.
 
ejmerkel said:
I tried setting the CPU type to Broadwell, stopping/starting the VM and then attempting to live migrate it. Unfortunately, the VM still locked up after the migration was complete.
Click to expand...
Thne I would give a newer kernel a shot
 
I have the same problem between a Dell and a supermicro server when I do a live migration the VM crashes. Did the 6.2 kernel fix the issue?
 
I have the same issue after an update to proxmox 8 today and the 6.8 kernel but I am pretty sure that the problem already was there with proxmox 7 also. I can migrate vms between two nodes of the cluster without a problem but migrating to the third node causes to crash the vms. I can post more infos on the hardware, just let me know what output of which command you need.

Processor on node1: Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz
Processor on node2: Intel(R) Xeon(R) W-2125 CPU @ 4.00GHz
Processor on node3: Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz

Migrating a vm from node2 to node1 or node3 causes a crash, all other variants seem to work.

We only use vms, no containers. In the vm configurations cpu is set to max.

Any ideas how I can get migrations to work for all nodes?
 
schoeppi said:
In the vm configurations cpu is set to max.
Click to expand...
What is max? If it's host then the crash is normal (see next comment), if not, please describe what max is.

schoeppi said:
Processor on node1: Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz
Processor on node2: Intel(R) Xeon(R) W-2125 CPU @ 4.00GHz
Processor on node3: Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz

Migrating a vm from node2 to node1 or node3 causes a crash, all other variants seem to work.
Click to expand...
Sounds reasonable. The VMs have most probably features enabled that are Xeon exclusive and do not exist on the i7s and the VM crashs. If you want a few to no problems, you identical hardware in a cluster.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom