Greetings,
EDIT: NVM, seems this is not related with restore procedure, but instead same problem as here: https://forum.proxmox.com/threads/memory-allocation-failure.41441/
I'm trying to restore old VM (KVM) created with Proxmox 3.4. Restore process went without any errors/warnings, but I can't run the VM with following error:
Kernel log (dmesg) shows a lot of weird info:
Any ideas?
EDIT: NVM, seems this is not related with restore procedure, but instead same problem as here: https://forum.proxmox.com/threads/memory-allocation-failure.41441/
I'm trying to restore old VM (KVM) created with Proxmox 3.4. Restore process went without any errors/warnings, but I can't run the VM with following error:
Code:
ioctl(KVM_CREATE_VM) failed: 12 Cannot allocate memory
kvm: failed to initialize KVM: Cannot allocate memory
TASK ERROR: start failed: command '/usr/bin/kvm -id 103 -name xxx -chardev 'socket,id=qmp,path=/var/run/qemu-server/103.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/103.pid -daemonize -smbios 'type=1,uuid=3bf19760-0eed-445b-abd7-df6c222c0199' -smp '1,sockets=1,cores=1,maxcpus=1' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vnc unix:/var/run/qemu-server/103.vnc,x509,password -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi -m 768 -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'VGA,id=vga,bus=pci.0,addr=0x2' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f67f7f2e16c6' -drive 'file=/dev/pve/vm-103-disk-0,if=none,id=drive-ide2,format=raw,cache=none,aio=native,detect-zeroes=on' -device 'ide-hd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=100' -drive 'if=none,id=drive-ide3,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=1,drive=drive-ide3,id=ide3,bootindex=200' -netdev 'type=tap,id=net0,ifname=tap103i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'e1000,mac=2E:A6:64:EF:88:CE,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -netdev 'type=tap,id=net1,ifname=tap103i1,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=4A:24:1F:F4:FB:66,netdev=net1,bus=pci.0,addr=0x13,id=net1,bootindex=301' -machine 'type=pc'' failed: exit code 1Kernel log (dmesg) shows a lot of weird info:
Code:
[Wed Feb 13 11:21:03 2019] kvm: page allocation failure: order:6, mode:0x140c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null)
[Wed Feb 13 11:21:03 2019] kvm cpuset=/ mems_allowed=0
[Wed Feb 13 11:21:03 2019] CPU: 1 PID: 21979 Comm: kvm Tainted: P O 4.15.18-8-pve #1
[Wed Feb 13 11:21:03 2019] Hardware name: Supermicro SSG-5029P-E1CTR12L/X11SPH-nCTF, BIOS 2.0b 02/26/2018
[Wed Feb 13 11:21:03 2019] Call Trace:
[Wed Feb 13 11:21:03 2019] dump_stack+0x63/0x8b
[Wed Feb 13 11:21:03 2019] warn_alloc+0xfc/0x190
[Wed Feb 13 11:21:03 2019] ? __alloc_pages_direct_compact+0xf4/0x100
[Wed Feb 13 11:21:03 2019] __alloc_pages_slowpath+0xefa/0xf10
[Wed Feb 13 11:21:03 2019] ? mntput+0x24/0x40
[Wed Feb 13 11:21:03 2019] ? terminate_walk+0x91/0xf0
[Wed Feb 13 11:21:03 2019] __alloc_pages_nodemask+0x25b/0x280
[Wed Feb 13 11:21:03 2019] alloc_pages_current+0x6a/0xe0
[Wed Feb 13 11:21:03 2019] kmalloc_order+0x18/0x40
[Wed Feb 13 11:21:03 2019] kmalloc_order_trace+0x24/0xa0
[Wed Feb 13 11:21:03 2019] ? kvm_vm_ioctl_check_extension+0xb0/0x140 [kvm]
[Wed Feb 13 11:21:03 2019] kvm_dev_ioctl+0xb5/0x6b0 [kvm]
[Wed Feb 13 11:21:03 2019] do_vfs_ioctl+0xa6/0x620
[Wed Feb 13 11:21:03 2019] ? putname+0x54/0x60
[Wed Feb 13 11:21:03 2019] ? do_sys_open+0x1bc/0x280
[Wed Feb 13 11:21:03 2019] SyS_ioctl+0x79/0x90
[Wed Feb 13 11:21:03 2019] do_syscall_64+0x73/0x130
[Wed Feb 13 11:21:03 2019] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[Wed Feb 13 11:21:03 2019] RIP: 0033:0x7f679ca07dd7
[Wed Feb 13 11:21:03 2019] RSP: 002b:00007ffc91865348 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[Wed Feb 13 11:21:03 2019] RAX: ffffffffffffffda RBX: 000000000000ae01 RCX: 00007f679ca07dd7
[Wed Feb 13 11:21:03 2019] RDX: 0000000000000000 RSI: 000000000000ae01 RDI: 000000000000000d
[Wed Feb 13 11:21:03 2019] RBP: 0000000000000000 R08: 00007f678ec616a0 R09: 0000000000000000
[Wed Feb 13 11:21:03 2019] R10: 0000000000000240 R11: 0000000000000246 R12: 00007f678edbe000
[Wed Feb 13 11:21:03 2019] R13: 00007f678ecec280 R14: 0000000000000000 R15: 00007f678edbe000
[Wed Feb 13 11:21:03 2019] Mem-Info:
[Wed Feb 13 11:21:03 2019] active_anon:2493951 inactive_anon:375739 isolated_anon:0
active_file:955073 inactive_file:664505 isolated_file:0
unevictable:5190 dirty:1288 writeback:410 unstable:0
slab_reclaimable:1062006 slab_unreclaimable:69850
mapped:259271 shmem:525171 pagetables:28285 bounce:0
free:68444 free_pcp:193 free_cma:0
[Wed Feb 13 11:21:03 2019] Node 0 active_anon:9975804kB inactive_anon:1502956kB active_file:3820292kB inactive_file:2658020kB unevictable:20760kB isolated(anon):0kB isolated(file):0kB mapped:1037084kB dirty:5152kB writeback:1640kB shmem:2100684kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[Wed Feb 13 11:21:03 2019] Node 0 DMA free:15884kB min:44kB low:56kB high:68kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15996kB managed:15884kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[Wed Feb 13 11:21:03 2019] lowmem_reserve[]: 0 1592 22702 22702 22702
[Wed Feb 13 11:21:03 2019] Node 0 DMA32 free:121760kB min:4736kB low:6364kB high:7992kB active_anon:219464kB inactive_anon:76956kB active_file:321812kB inactive_file:115800kB unevictable:0kB writepending:864kB present:1732864kB managed:1667292kB mlocked:0kB kernel_stack:208kB pagetables:8288kB bounce:0kB free_pcp:36kB local_pcp:0kB free_cma:0kB
[Wed Feb 13 11:21:03 2019] lowmem_reserve[]: 0 0 21109 21109 21109
[Wed Feb 13 11:21:03 2019] Node 0 Normal free:136132kB min:62796kB low:84408kB high:106020kB active_anon:9756788kB inactive_anon:1425776kB active_file:3498116kB inactive_file:2542456kB unevictable:20760kB writepending:5928kB present:22020096kB managed:21622144kB mlocked:20820kB kernel_stack:9040kB pagetables:104852kB bounce:0kB free_pcp:736kB local_pcp:4kB free_cma:0kB
[Wed Feb 13 11:21:03 2019] lowmem_reserve[]: 0 0 0 0 0
[Wed Feb 13 11:21:03 2019] Node 0 DMA: 1*4kB (U) 1*8kB (U) 2*16kB (U) 1*32kB (U) 3*64kB (U) 0*128kB 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15884kB
[Wed Feb 13 11:21:03 2019] Node 0 DMA32: 2067*4kB (UMEH) 702*8kB (UMEH) 442*16kB (UMEH) 2825*32kB (UMEH) 79*64kB (UEH) 28*128kB (H) 7*256kB (H) 1*512kB (H) 0*1024kB 0*2048kB 0*4096kB = 122300kB
[Wed Feb 13 11:21:03 2019] Node 0 Normal: 13731*4kB (UMEH) 3512*8kB (UMEH) 3413*16kB (UEH) 15*32kB (UH) 6*64kB (H) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 138492kB
[Wed Feb 13 11:21:03 2019] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[Wed Feb 13 11:21:03 2019] Node 0 hugepages_total=32 hugepages_free=14 hugepages_surp=0 hugepages_size=2048kB
[Wed Feb 13 11:21:03 2019] 2147895 total pagecache pages
[Wed Feb 13 11:21:03 2019] 1441 pages in swap cache
[Wed Feb 13 11:21:03 2019] Swap cache stats: add 4774935, delete 4773284, find 320887541/322227999
[Wed Feb 13 11:21:03 2019] Free swap = 244564kB
[Wed Feb 13 11:21:03 2019] Total swap = 1048572kB
[Wed Feb 13 11:21:03 2019] 5942239 pages RAM
[Wed Feb 13 11:21:03 2019] 0 pages HighMem/MovableOnly
[Wed Feb 13 11:21:03 2019] 115909 pages reserved
[Wed Feb 13 11:21:03 2019] 0 pages cma reserved
[Wed Feb 13 11:21:03 2019] 0 pages hwpoisoned
[Wed Feb 13 11:21:05 2019] kvm: page allocation failure: order:6, mode:0x140c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null)
[Wed Feb 13 11:21:05 2019] kvm cpuset=/ mems_allowed=0
[Wed Feb 13 11:21:05 2019] CPU: 13 PID: 21995 Comm: kvm Tainted: P O 4.15.18-8-pve #1
[Wed Feb 13 11:21:05 2019] Hardware name: Supermicro SSG-5029P-E1CTR12L/X11SPH-nCTF, BIOS 2.0b 02/26/2018
[Wed Feb 13 11:21:05 2019] Call Trace:
[Wed Feb 13 11:21:05 2019] dump_stack+0x63/0x8b
[Wed Feb 13 11:21:05 2019] warn_alloc+0xfc/0x190
[Wed Feb 13 11:21:05 2019] ? __alloc_pages_direct_compact+0x51/0x100
[Wed Feb 13 11:21:05 2019] __alloc_pages_slowpath+0xefa/0xf10
[Wed Feb 13 11:21:05 2019] ? mntput+0x24/0x40
[Wed Feb 13 11:21:05 2019] ? terminate_walk+0x91/0xf0
[Wed Feb 13 11:21:05 2019] __alloc_pages_nodemask+0x25b/0x280
[Wed Feb 13 11:21:05 2019] alloc_pages_current+0x6a/0xe0
[Wed Feb 13 11:21:05 2019] kmalloc_order+0x18/0x40
[Wed Feb 13 11:21:05 2019] kmalloc_order_trace+0x24/0xa0
[Wed Feb 13 11:21:05 2019] ? kvm_vm_ioctl_check_extension+0xb0/0x140 [kvm]
[Wed Feb 13 11:21:05 2019] kvm_dev_ioctl+0xb5/0x6b0 [kvm]
[Wed Feb 13 11:21:05 2019] do_vfs_ioctl+0xa6/0x620
[Wed Feb 13 11:21:05 2019] ? putname+0x54/0x60
[Wed Feb 13 11:21:05 2019] ? do_sys_open+0x1bc/0x280
[Wed Feb 13 11:21:05 2019] SyS_ioctl+0x79/0x90
[Wed Feb 13 11:21:05 2019] do_syscall_64+0x73/0x130
[Wed Feb 13 11:21:05 2019] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[Wed Feb 13 11:21:05 2019] RIP: 0033:0x7fe61c71add7
[Wed Feb 13 11:21:05 2019] RSP: 002b:00007ffd0b973be8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[Wed Feb 13 11:21:05 2019] RAX: ffffffffffffffda RBX: 000000000000ae01 RCX: 00007fe61c71add7
[Wed Feb 13 11:21:05 2019] RDX: 0000000000000000 RSI: 000000000000ae01 RDI: 000000000000000d
[Wed Feb 13 11:21:05 2019] RBP: 0000000000000000 R08: 00007fe60ec616a0 R09: 0000000000000000
[Wed Feb 13 11:21:05 2019] R10: 0000000000000240 R11: 0000000000000246 R12: 00007fe60edbe000
[Wed Feb 13 11:21:05 2019] R13: 00007fe60ecec280 R14: 0000000000000000 R15: 00007fe60edbe000
[Wed Feb 13 11:21:05 2019] Mem-Info:
[Wed Feb 13 11:21:05 2019] active_anon:2493303 inactive_anon:364635 isolated_anon:0
active_file:955441 inactive_file:664116 isolated_file:0
unevictable:5190 dirty:1393 writeback:0 unstable:0
slab_reclaimable:1061809 slab_unreclaimable:69917
mapped:259369 shmem:513920 pagetables:28093 bounce:0
free:80674 free_pcp:246 free_cma:0
[Wed Feb 13 11:21:05 2019] Node 0 active_anon:9973212kB inactive_anon:1458540kB active_file:3821764kB inactive_file:2656464kB unevictable:20760kB isolated(anon):0kB isolated(file):0kB mapped:1037476kB dirty:5572kB writeback:0kB shmem:2055680kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[Wed Feb 13 11:21:05 2019] Node 0 DMA free:15884kB min:44kB low:56kB high:68kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15996kB managed:15884kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[Wed Feb 13 11:21:05 2019] lowmem_reserve[]: 0 1592 22702 22702 22702
[Wed Feb 13 11:21:05 2019] Node 0 DMA32 free:140996kB min:4736kB low:6364kB high:7992kB active_anon:219468kB inactive_anon:59008kB active_file:321932kB inactive_file:115692kB unevictable:0kB writepending:704kB present:1732864kB managed:1667292kB mlocked:0kB kernel_stack:208kB pagetables:8288kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[Wed Feb 13 11:21:05 2019] lowmem_reserve[]: 0 0 21109 21109 21109
[Wed Feb 13 11:21:05 2019] Node 0 Normal free:165816kB min:62796kB low:84408kB high:106020kB active_anon:9753872kB inactive_anon:1399448kB active_file:3499832kB inactive_file:2540852kB unevictable:20760kB writepending:4868kB present:22020096kB managed:21622144kB mlocked:20820kB kernel_stack:9072kB pagetables:104084kB bounce:0kB free_pcp:984kB local_pcp:0kB free_cma:0kB
[Wed Feb 13 11:21:05 2019] lowmem_reserve[]: 0 0 0 0 0
[Wed Feb 13 11:21:05 2019] Node 0 DMA: 1*4kB (U) 1*8kB (U) 2*16kB (U) 1*32kB (U) 3*64kB (U) 0*128kB 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15884kB
[Wed Feb 13 11:21:05 2019] Node 0 DMA32: 2731*4kB (UMEH) 1114*8kB (UMEH) 732*16kB (UMEH) 3083*32kB (UMEH) 78*64kB (UEH) 28*128kB (H) 7*256kB (H) 1*512kB (H) 0*1024kB 0*2048kB 0*4096kB = 141084kB
[Wed Feb 13 11:21:05 2019] Node 0 Normal: 14899*4kB (UMEH) 6292*8kB (UMEH) 3518*16kB (UEH) 10*32kB (H) 6*64kB (H) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 166924kB
[Wed Feb 13 11:21:05 2019] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[Wed Feb 13 11:21:05 2019] Node 0 hugepages_total=32 hugepages_free=14 hugepages_surp=0 hugepages_size=2048kB
[Wed Feb 13 11:21:05 2019] 2136652 total pagecache pages
[Wed Feb 13 11:21:05 2019] 1436 pages in swap cache
[Wed Feb 13 11:21:05 2019] Swap cache stats: add 4774943, delete 4773297, find 320887644/322228110
[Wed Feb 13 11:21:05 2019] Free swap = 244564kB
[Wed Feb 13 11:21:05 2019] Total swap = 1048572kB
[Wed Feb 13 11:21:05 2019] 5942239 pages RAM
[Wed Feb 13 11:21:05 2019] 0 pages HighMem/MovableOnly
[Wed Feb 13 11:21:05 2019] 115909 pages reserved
[Wed Feb 13 11:21:05 2019] 0 pages cma reserved
[Wed Feb 13 11:21:05 2019] 0 pages hwpoisonedAny ideas?