Strange migration behavior

SergeyMorozov

Member
Sep 21, 2020
18
8
23
43
Hello, in my home lab I found some strange behavior during migration. I trying live migration between two hosts: Ryzen 5 5800x and Celeron 5105. Celeron -> Ryzen works perfectly, but not in the opposite direction.

If I trying migrate a vm with UEFI and x86_64-v2 CPU (I tried kvm64 and qemu64 with same results) from Ryzen to Celeron, VM is crashing.
[ 71.004996] BUG: unable to handle page fault for address: ffffa8cc000b9006
[ 71.006624] #PF: supervisor read access in kernel mode
[ 71.007358] #PF: error_code(0x0009) - reserved bit violation
[ 71.007986] PGD 1800067 P4D 1800067 PUD 1a6f067 PMD 1a70067 PTE 800000c06000a173
[ 71.009653] Oops: 0009 [#1] PREEMPT SMP NOPTI
[ 71.010268] CPU: 0 PID: 368 Comm: kworker/0:3 Not tainted 6.8.0-51-generic #52-Ubuntu
[ 71.011650] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 4.2023.08-1 11/07/2023
[ 71.013931] Workqueue: events virtnet_config_changed_work
[ 71.015685] RIP: 0010:ioread16+0x3e/0x90
[ 71.016403] Code: 66 ed 0f b7 c0 31 d2 31 f6 31 ff c3 cc cc cc cc 8b 05 56 c8 e4 01 85 c0 75 21 b8 ff ff 00 00 31 d2 31 f6 31 ff c3 cc cc cc cc <66> 8b 07 0f b7 c0 31 d2 31 f6 31 ff c3 cc cc cc cc 55 83 e8 01 48
[ 71.019891] RSP: 0018:ffffa8cc0072bde8 EFLAGS: 00010292
[ 71.021008] RAX: 0000000000000008 RBX: ffff8b24c2b0da90 RCX: 0000000000000002
[ 71.022588] RDX: ffffa8cc0072be1e RSI: 0000000000000006 RDI: ffffa8cc000b9006
[ 71.024422] RBP: ffffa8cc0072be08 R08: 0000000000000000 R09: 0000000000000000
[ 71.025981] R10: 0000000000000000 R11: 0000000000000000 R12: ffffa8cc0072be1e
[ 71.027604] R13: ffffa8cc000b9000 R14: ffff8b24c19d3205 R15: ffff8b24c2b0da90
[ 71.029887] FS: 0000000000000000(0000) GS:ffff8b24fc800000(0000) knlGS:0000000000000000
[ 71.031266] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 71.032100] CR2: ffffa8cc000b9006 CR3: 00000000355b2000 CR4: 00000000000006f0
[ 71.036023] Call Trace:
[ 71.037263] <TASK>
[ 71.038274] ? show_regs+0x6d/0x80
[ 71.038754] ? __die+0x24/0x80
[ 71.039170] ? page_fault_oops+0x99/0x1b0
[ 71.039741] ? kernelmode_fixup_or_oops.isra.0+0x69/0x90
[ 71.040918] ? __bad_area_nosemaphore+0x19d/0x2c0
[ 71.043134] ? sched_clock_cpu+0x12/0x1e0
[ 71.044811] ? bad_area_nosemaphore+0x16/0x30
[ 71.045607] ? do_kern_addr_fault+0x7b/0xa0
[ 71.047052] ? exc_page_fault+0x1a4/0x1b0
[ 71.047663] ? asm_exc_page_fault+0x27/0x30
[ 71.048226] ? ioread16+0x3e/0x90
[ 71.049014] ? vp_get+0x4d/0x100
[ 71.049958] virtnet_config_changed_work+0xa5/0x1a0
[ 71.050907] process_one_work+0x178/0x350
[ 71.052084] worker_thread+0x306/0x440
[ 71.052569] ? _raw_spin_unlock_irqrestore+0x11/0x60
[ 71.053240] ? __pfx_worker_thread+0x10/0x10
[ 71.054727] kthread+0xf2/0x120
[ 71.055756] ? __pfx_kthread+0x10/0x10
[ 71.056380] ret_from_fork+0x47/0x70
[ 71.057565] ? __pfx_kthread+0x10/0x10
[ 71.058202] ret_from_fork_asm+0x1b/0x30
[ 71.058761] </TASK>
[ 71.059041] Modules linked in: qrtr cfg80211 binfmt_misc i2c_piix4 nls_iso8859_1 vmgenid joydev input_leds mac_hid serio_raw dm_multipath msr efi_pstore nfnetlink dmi_sysfs qemu_fw_cfg ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 sha256_ssse3 sha1_ssse3 psmouse floppy pata_acpi crypto_simd cryptd
[ 71.065506] CR2: ffffa8cc000b9006
[ 71.065941] ---[ end trace 0000000000000000 ]---
[ 71.776426] RIP: 0010:ioread16+0x3e/0x90
[ 71.777529] Code: 66 ed 0f b7 c0 31 d2 31 f6 31 ff c3 cc cc cc cc 8b 05 56 c8 e4 01 85 c0 75 21 b8 ff ff 00 00 31 d2 31 f6 31 ff c3 cc cc cc cc <66> 8b 07 0f b7 c0 31 d2 31 f6 31 ff c3 cc cc cc cc 55 83 e8 01 48
[ 71.779935] RSP: 0018:ffffa8cc0072bde8 EFLAGS: 00010292
[ 71.780681] RAX: 0000000000000008 RBX: ffff8b24c2b0da90 RCX: 0000000000000002
[ 71.781643] RDX: ffffa8cc0072be1e RSI: 0000000000000006 RDI: ffffa8cc000b9006
[ 71.782630] RBP: ffffa8cc0072be08 R08: 0000000000000000 R09: 0000000000000000
[ 71.783602] R10: 0000000000000000 R11: 0000000000000000 R12: ffffa8cc0072be1e
[ 71.784696] R13: ffffa8cc000b9000 R14: ffff8b24c19d3205 R15: ffff8b24c2b0da90
[ 71.785951] FS: 0000000000000000(0000) GS:ffff8b24fc800000(0000) knlGS:0000000000000000
[ 71.787071] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 71.787917] CR2: ffffa8cc000b9006 CR3: 00000000355b2000 CR4: 00000000000006f0
[ 71.789088] note: kworker/0:3[368] exited with irqs disabled
[ 71.794431] BUG: unable to handle page fault for address: ffffffffb86c2ae1
[ 71.795495] #PF: supervisor write access in kernel mode
[ 71.796363] #PF: error_code(0x0003) - permissions violation
[ 71.797219] PGD 7a41067 P4D 7a41067 PUD 7a42063 PMD 56001a1
[ 71.798134] Oops: 0003 [#2] PREEMPT SMP NOPTI
[ 71.798845] CPU: 1 PID: 292 Comm: systemd-journal Tainted: G D 6.8.0-51-generic #52-Ubuntu
[ 71.800445] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 4.2023.08-1 11/07/2023
[ 71.801667] RIP: 0010:kvm_kick_cpu+0x61/0x90
[ 71.802270] Code: bf 05 00 00 00 e8 4f 17 f4 ff 5b 41 5c 5d 31 c0 31 d2 31 c9 31 f6 31 ff 45 31 c0 c3 cc cc cc cc b8 05 00 00 00 31 db 48 89 d1 <0f> 01 d9 5b 41 5c 5d 31 c0 31 d2 31 c9 31 f6 31 ff 45 31 c0 c3 cc
[ 71.805759] RSP: 0000:ffffa8cc00307cd8 EFLAGS: 00010046
[ 71.806570] RAX: 0000000000000005 RBX: 0000000000000000 RCX: 0000000000000000
[ 71.807548] RDX: 0000000000000000 RSI: 00000000000000f8 RDI: 0000000000000000
[ 71.808904] RBP: ffffa8cc00307ce8 R08: ffff8b24ffe3b040 R09: 00000000000000ff
[ 71.809924] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000019a34
[ 71.810903] R13: 0000000000000287 R14: 0000000000000002 R15: ffffffffbaa06108
[ 71.811750] FS: 00007ac859198440(0000) GS:ffff8b24fc900000(0000) knlGS:0000000000000000
[ 71.812753] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 71.813431] CR2: ffffffffb86c2ae1 CR3: 00000000357f8000 CR4: 00000000000006f0
[ 71.814256] Call Trace:
[ 71.814594] <TASK>
[ 71.814864] ? show_regs+0x6d/0x80
[ 71.815309] ? __die+0x24/0x80
[ 71.815808] ? kvm_kick_cpu+0x61/0x90
[ 71.816325] ? page_fault_oops+0x99/0x1b0
[ 71.816858] ? kvm_kick_cpu+0x61/0x90
[ 71.817319] ? kernelmode_fixup_or_oops.isra.0+0x69/0x90
[ 71.817962] ? kvm_kick_cpu+0x61/0x90
[ 71.818419] ? __bad_area_nosemaphore+0x19d/0x2c0
[ 71.819171] ? kvm_kick_cpu+0x61/0x90
[ 71.819747] ? bad_area_nosemaphore+0x16/0x30
[ 71.820439] ? do_kern_addr_fault+0x7b/0xa0
[ 71.820989] ? kvm_kick_cpu+0x61/0x90
[ 71.821471] ? exc_page_fault+0x1a4/0x1b0
[ 71.822044] ? asm_exc_page_fault+0x27/0x30
[ 71.822856] ? kvm_kick_cpu+0x61/0x90
[ 71.823306] __pv_queued_spin_unlock_slowpath+0xaf/0x150
[ 71.823971] __raw_callee_save___pv_queued_spin_unlock_slowpath+0x15/0x30
[ 71.824901] .slowpath+0x9/0x15
[ 71.826000] _raw_spin_unlock_irqrestore+0x11/0x60
[ 71.828223] swake_up_one+0x5f/0x80
[ 71.828791] swake_up_one_online+0x38/0xa0
[ 71.829308] rcu_gp_kthread_wake+0x57/0x90
[ 71.829878] rcu_report_qs_rnp+0x119/0x140
[ 71.830392] rcu_report_qs_rdp+0xa7/0x180
[ 71.830894] rcu_core+0xcb/0x390
[ 71.831320] rcu_core_si+0xe/0x20
[ 71.831736] handle_softirqs+0xdb/0x340
[ 71.832215] __irq_exit_rcu+0xd9/0x100
[ 71.832737] irq_exit_rcu+0xe/0x20
[ 71.833209] sysvec_apic_timer_interrupt+0x40/0xd0
[ 71.833802] asm_sysvec_apic_timer_interrupt+0x1b/0x20
[ 71.834679] RIP: 0033:0x7ac858a5f0b0
[ 71.835211] Code: cc cc cc cc 66 0f 1f 44 00 00 48 89 f8 48 89 f9 48 83 ee 01 49 f7 e1 48 c1 ea 03 48 8d 04 92 48 01 c0 48 29 c1 41 0f b6 04 08 <88> 06 48 89 f8 48 89 d7 48 83 f8 09 77 d2 48 89 f0 c3 66 66 2e 0f
[ 71.837577] RSP: 002b:00007ffe82167428 EFLAGS: 00000212
[ 71.838387] RAX: 0000000000000038 RBX: 0000000000000001 RCX: 0000000000000008
[ 71.839323] RDX: 00000000006c5e94 RSI: 00007ffe82167917 RDI: 00000000043bb1d0
[ 71.840334] RBP: 00007ffe82167950 R08: 00007ac858bcfbe0 R09: cccccccccccccccd
[ 71.841265] R10: 0000000000000001 R11: 0000000000000000 R12: 00000000043bb1d0
[ 71.842159] R13: 00007ffe82167960 R14: 00007ffe82167aa0 R15: 00006170796b3c27
[ 71.843054] </TASK>
[ 71.843356] Modules linked in: qrtr cfg80211 binfmt_misc i2c_piix4 nls_iso8859_1 vmgenid joydev input_leds mac_hid serio_raw dm_multipath msr efi_pstore nfnetlink dmi_sysfs qemu_fw_cfg ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 sha256_ssse3 sha1_ssse3 psmouse floppy pata_acpi crypto_simd cryptd
[ 71.847876] CR2: ffffffffb86c2ae1
[ 71.848334] ---[ end trace 0000000000000000 ]---
[ 72.553315] RIP: 0010:ioread16+0x3e/0x90
[ 72.553944] Code: 66 ed 0f b7 c0 31 d2 31 f6 31 ff c3 cc cc cc cc 8b 05 56 c8 e4 01 85 c0 75 21 b8 ff ff 00 00 31 d2 31 f6 31 ff c3 cc cc cc cc <66> 8b 07 0f b7 c0 31 d2 31 f6 31 ff c3 cc cc cc cc 55 83 e8 01 48
[ 72.556298] RSP: 0018:ffffa8cc0072bde8 EFLAGS: 00010292
[ 72.556932] RAX: 0000000000000008 RBX: ffff8b24c2b0da90 RCX: 0000000000000002
[ 72.557825] RDX: ffffa8cc0072be1e RSI: 0000000000000006 RDI: ffffa8cc000b9006
[ 72.558750] RBP: ffffa8cc0072be08 R08: 0000000000000000 R09: 0000000000000000
[ 72.559885] R10: 0000000000000000 R11: 0000000000000000 R12: ffffa8cc0072be1e
[ 72.560802] R13: ffffa8cc000b9000 R14: ffff8b24c19d3205 R15: ffff8b24c2b0da90
[ 72.561830] FS: 00007ac859198440(0000) GS:ffff8b24fc900000(0000) knlGS:0000000000000000
[ 72.562848] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 72.563587] CR2: ffffffffb86c2ae1 CR3: 00000000357f8000 CR4: 00000000000006f0
[ 72.564533] Kernel panic - not syncing: Fatal exception in interrupt
[ 73.712550] Shutting down cpus with NMI
[ 73.713327] Kernel Offset: 0x37600000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 74.458191] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
VM is crashing only if BIOS set to OVMF (UEFI), migration of VM with SeaBIOS is working fine. But if I change CPU from x86_64-v2 (or from kvm64 / qemu64) to Westmere, the migration starts working again.
agent: 1
bios: ovmf
boot: order=scsi0;ide2;net0
cores: 2
cpu: x86-64-v2
efidisk0: local-zfs:vm-503-disk-0,efitype=4m,pre-enrolled-keys=1,size=528K
ide2: none,media=cdrom
memory: 1024
meta: creation-qemu=9.0.2,ctime=1736930445
name: test-uefi
net0: virtio=BC:24:11:A6:FF:7A,bridge=lab,firewall=1
numa: 0
ostype: l26
scsi0: local-zfs:vm-503-disk-1,discard=on,iothread=1,size=10G
scsihw: virtio-scsi-single
serial0: socket
smbios1: uuid=abf89aac-df60-4a42-b182-4081a8455d5e
sockets: 1
vga: serial0
vmgenid: 2787de05-a2b3-42af-8d7f-636e6cf699bb

Results of migration attempt with different VM settings from Ryzen to Celeron:
VM firmware typeVM processormigration results
OVMF (UEFI)x86_64-v2, qemu64, kvm64fail
OVMF (UEFI)Westmereok
SeaBIOSx86_64-v2, qemu64, kvm64ok

Looks like x86_64-v2 cpu is different on AMD and Intel host platforms, but only in UEFI vm.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!