[SOLVED] VMs starten nicht nach Update (kvm)

pxo

Renowned Member
Nov 3, 2013
29
0
66
Seit dem heutigen Update (21.04)

Code:
Start-Date: 2018-03-12  11:27:44
Commandline: apt-get dist-upgrade
Install: proxmox-widget-toolkit:amd64 (1.0-11, automatic), pve-i18n:amd64 (1.0-4, automatic)
Upgrade: postfix:amd64 (3.1.6-0+deb9u1, 3.1.8-0+deb9u1), libpve-access-control:amd64 (5.0-7, 5.0-8), libcups2:amd64 (2.2.1-8, 2.2.1-8+deb9u1), libdbus-1-3:amd64 (1.10.24-0+deb9u1, 1.10.26-0+deb9u1), libfdisk1:amd64 (2.29.2-1, 2.29.2-1+deb9u1), publicsuffix:amd64 (20171028.2055-0+deb9u1, 20180218.2049-0+deb9u1), linux-libc-dev:amd64 (4.9.65-3+deb9u2, 4.9.82-1+deb9u3), libc6-dev:amd64 (2.24-11+deb9u1, 2.24-11+deb9u3), libsystemd0:amd64 (232-25+deb9u1, 232-25+deb9u2), cpp-6:amd64 (6.3.0-18, 6.3.0-18+deb9u1), dbus:amd64 (1.10.24-0+deb9u1, 1.10.26-0+deb9u1), libmount1:amd64 (2.29.2-1, 2.29.2-1+deb9u1), postfix-sqlite:amd64 (3.1.6-0+deb9u1, 3.1.8-0+deb9u1), pve-qemu-kvm:amd64 (2.9.1-6, 2.9.1-9), libncurses5:amd64 (6.0+20161126-1+deb9u1, 6.0+20161126-1+deb9u2), libc6:amd64 (2.24-11+deb9u1, 2.24-11+deb9u3), util-linux:amd64 (2.29.2-1, 2.29.2-1+deb9u1), pve-ha-manager:amd64 (2.0-4, 2.0-5), openssh-sftp-server:amd64 (1:7.4p1-10+deb9u2, 1:7.4p1-10+deb9u3), udev:amd64 (232-25+deb9u1, 232-25+deb9u2), locales:amd64 (2.24-11+deb9u1, 2.24-11+deb9u3), pve-container:amd64 (2.0-18, 2.0-19), libquadmath0:amd64 (6.3.0-18, 6.3.0-18+deb9u1), isc-dhcp-common:amd64 (4.3.5-3, 4.3.5-3+deb9u1), gcc-6-base:amd64 (6.3.0-18, 6.3.0-18+deb9u1), libncursesw5:amd64 (6.0+20161126-1+deb9u1, 6.0+20161126-1+deb9u2), pve-cluster:amd64 (5.0-19, 5.0-20), libudev1:amd64 (232-25+deb9u1, 232-25+deb9u2), libapparmor1:amd64 (2.11.0-3, 2.11.0-3+deb9u2), libgcc1:amd64 (1:6.3.0-18, 1:6.3.0-18+deb9u1), mount:amd64 (2.29.2-1, 2.29.2-1+deb9u1), libblkid1:amd64 (2.29.2-1, 2.29.2-1+deb9u1), libc-l10n:amd64 (2.24-11+deb9u1, 2.24-11+deb9u3), libapparmor-perl:amd64 (2.11.0-3, 2.11.0-3+deb9u2), libc-bin:amd64 (2.24-11+deb9u1, 2.24-11+deb9u3), pve-manager:amd64 (5.1-43, 5.1-46), libtinfo5:amd64 (6.0+20161126-1+deb9u1, 6.0+20161126-1+deb9u2), reportbug:amd64 (7.1.7, 7.1.7+deb9u1), libvorbisenc2:amd64 (1.3.5-4, 1.3.5-4+deb9u1), libgfortran3:amd64 (6.3.0-18, 6.3.0-18+deb9u1), ssh:amd64 (1:7.4p1-10+deb9u2, 1:7.4p1-10+deb9u3), libzfs2linux:amd64 (0.7.4-pve2~bpo9, 0.7.6-pve1~bpo9), systemd-sysv:amd64 (232-25+deb9u1, 232-25+deb9u2), libuuid1:amd64 (2.29.2-1, 2.29.2-1+deb9u1), libpve-common-perl:amd64 (5.0-25, 5.0-28), w3m:amd64 (0.5.3-34, 0.5.3-34+deb9u1), libpam-systemd:amd64 (232-25+deb9u1, 232-25+deb9u2), lxc-pve:amd64 (2.1.1-2, 2.1.1-3), ncurses-term:amd64 (6.0+20161126-1+deb9u1, 6.0+20161126-1+deb9u2), systemd:amd64 (232-25+deb9u1, 232-25+deb9u2), qemu-server:amd64 (5.0-20, 5.0-22), libsmartcols1:amd64 (2.29.2-1, 2.29.2-1+deb9u1), python3-reportbug:amd64 (7.1.7, 7.1.7+deb9u1), ncurses-bin:amd64 (6.0+20161126-1+deb9u1, 6.0+20161126-1+deb9u2), openssh-server:amd64 (1:7.4p1-10+deb9u2, 1:7.4p1-10+deb9u3), openssh-client:amd64 (1:7.4p1-10+deb9u2, 1:7.4p1-10+deb9u3), bsdutils:amd64 (1:2.29.2-1, 1:2.29.2-1+deb9u1), ncurses-base:amd64 (6.0+20161126-1+deb9u1, 6.0+20161126-1+deb9u2), libzpool2linux:amd64 (0.7.4-pve2~bpo9, 0.7.6-pve1~bpo9), libdirectfb-1.2-9:amd64 (1.2.10.0-8, 1.2.10.0-8+deb9u1), libc-dev-bin:amd64 (2.24-11+deb9u1, 2.24-11+deb9u3), multiarch-support:amd64 (2.24-11+deb9u1, 2.24-11+deb9u3), apparmor:amd64 (2.11.0-3, 2.11.0-3+deb9u2), cron:amd64 (3.0pl1-128+b1, 3.0pl1-128+deb9u1), libvorbis0a:amd64 (1.3.5-4, 1.3.5-4+deb9u1), libnvpair1linux:amd64 (0.7.4-pve2~bpo9, 0.7.6-pve1~bpo9), libuutil1linux:amd64 (0.7.4-pve2~bpo9, 0.7.6-pve1~bpo9), libstdc++6:amd64 (6.3.0-18, 6.3.0-18+deb9u1), isc-dhcp-client:amd64 (4.3.5-3, 4.3.5-3+deb9u1), base-files:amd64 (9.9+deb9u3, 9.9+deb9u4), lxcfs:amd64 (2.0.8-1, 2.0.8-2)
End-Date: 2018-03-12  11:29:46

Start-Date: 2018-04-21  00:45:51
Commandline: apt-get dist-upgrade
Install: ebtables:amd64 (2.0.10.4-3.5+b1, automatic), genisoimage:amd64 (9:1.1.11-3+b2, automatic), pve-edk2-firmware:amd64 (1.20180316-1, automatic), libpve-apiclient-perl:amd64 (2.0-4, automatic)
Upgrade: proxmox-widget-toolkit:amd64 (1.0-11, 1.0-15), corosync:amd64 (2.4.2-pve3, 2.4.2-pve4), perl-base:amd64 (5.24.1-3+deb9u2, 5.24.1-3+deb9u3), libcmap4:amd64 (2.4.2-pve3, 2.4.2-pve4), libpve-storage-perl:amd64 (5.0-17, 5.0-18), openssl:amd64 (1.1.0f-3+deb9u1, 1.1.0f-3+deb9u2), libsystemd0:amd64 (232-25+deb9u2, 232-25+deb9u3), libquorum5:amd64 (2.4.2-pve3, 2.4.2-pve4), pve-qemu-kvm:amd64 (2.9.1-9, 2.11.1-5), libcorosync4-pve:amd64 (2.4.2-pve3, 2.4.2-pve4), perl-modules-5.24:amd64 (5.24.1-3+deb9u2, 5.24.1-3+deb9u3), pve-docs:amd64 (5.1-16, 5.1-17), libtotem-pg5:amd64 (2.4.2-pve3, 2.4.2-pve4), pve-firewall:amd64 (3.0-5, 3.0-8), udev:amd64 (232-25+deb9u2, 232-25+deb9u3), pve-container:amd64 (2.0-19, 2.0-21), pve-cluster:amd64 (5.0-20, 5.0-25), libudev1:amd64 (232-25+deb9u2, 232-25+deb9u3), librados2-perl:amd64 (1.0-4, 1.0-5), libperl5.24:amd64 (5.24.1-3+deb9u2, 5.24.1-3+deb9u3), pve-manager:amd64 (5.1-46, 5.1-51), libvotequorum8:amd64 (2.4.2-pve3, 2.4.2-pve4), libzfs2linux:amd64 (0.7.6-pve1~bpo9, 0.7.7-pve1~bpo9), systemd-sysv:amd64 (232-25+deb9u2, 232-25+deb9u3), libpve-common-perl:amd64 (5.0-28, 5.0-30), libpam-systemd:amd64 (232-25+deb9u2, 232-25+deb9u3), lxc-pve:amd64 (2.1.1-3, 3.0.0-2), systemd:amd64 (232-25+deb9u2, 232-25+deb9u3), qemu-server:amd64 (5.0-22, 5.0-25), libcfg6:amd64 (2.4.2-pve3, 2.4.2-pve4), libzpool2linux:amd64 (0.7.6-pve1~bpo9, 0.7.7-pve1~bpo9), perl:amd64 (5.24.1-3+deb9u2, 5.24.1-3+deb9u3), libssl1.1:amd64 (1.1.0f-3+deb9u1, 1.1.0f-3+deb9u2), libnvpair1linux:amd64 (0.7.6-pve1~bpo9, 0.7.7-pve1~bpo9), libuutil1linux:amd64 (0.7.6-pve1~bpo9, 0.7.7-pve1~bpo9), libcpg4:amd64 (2.4.2-pve3, 2.4.2-pve4), libcorosync-common4:amd64 (2.4.2-pve3, 2.4.2-pve4), libssl1.0.2:amd64 (1.0.2l-2+deb9u2, 1.0.2l-2+deb9u3), tzdata:amd64 (2018c-0+deb9u1, 2018d-0+deb9u1), lxcfs:amd64 (2.0.8-2, 3.0.0-1)
End-Date: 2018-04-21  00:47:21


..starten auf dem node 3 von 4 kvm-vms nicht mehr. (Kann keine relevanten Konfigurationsunterschiede zwischen den VMs erkennen)
Die Meldungen beim Start von qm start 102:

Code:
Apr 21 05:38:28 px1 qm[4805]: <root@pam> starting task UPID:px1:000012D5:0000476A:5ADAB234:qmstart:102:root@pam:
Apr 21 05:38:28 px1 qm[4821]: start VM 102: UPID:px1:000012D5:0000476A:5ADAB234:qmstart:102:root@pam:
Apr 21 05:38:28 px1 systemd[1]: Created slice qemu.slice.
Apr 21 05:38:28 px1 systemd[1]: Started 102.scope.
Apr 21 05:38:29 px1 kernel: [  183.178857] device tap102i0 entered promiscuous mode
Apr 21 05:38:29 px1 kernel: [  183.185243] vmbr0: port 2(tap102i0) entered blocking state
Apr 21 05:38:29 px1 kernel: [  183.185244] vmbr0: port 2(tap102i0) entered disabled state
Apr 21 05:38:29 px1 kernel: [  183.185310] vmbr0: port 2(tap102i0) entered blocking state
Apr 21 05:38:29 px1 kernel: [  183.185311] vmbr0: port 2(tap102i0) entered forwarding state
Apr 21 05:38:29 px1 qm[4805]: <root@pam> end task UPID:px1:000012D5:0000476A:5ADAB234:qmstart:102:root@pam: OK
Apr 21 05:38:30 px1 pvestatd[1461]: storage 'kvm' is not online
Apr 21 05:38:40 px1 pvestatd[1461]: storage 'kvm' is not online
Apr 21 05:38:43 px1 systemd[1]: Created slice User Slice of prtg.
Apr 21 05:38:43 px1 systemd[1]: Starting User Manager for UID 1000...
Apr 21 05:38:43 px1 systemd[1]: Started Session 12 of user prtg.
Apr 21 05:38:43 px1 systemd[5086]: Reached target Paths.
Apr 21 05:38:43 px1 systemd[5086]: Listening on GnuPG cryptographic agent and passphrase cache.
Apr 21 05:38:43 px1 systemd[5086]: Reached target Timers.
Apr 21 05:38:43 px1 systemd[5086]: Listening on GnuPG network certificate management daemon.
Apr 21 05:38:43 px1 systemd[5086]: Listening on GnuPG cryptographic agent (ssh-agent emulation).
Apr 21 05:38:43 px1 systemd[5086]: Listening on GnuPG cryptographic agent and passphrase cache (restricted).
Apr 21 05:38:43 px1 systemd[5086]: Listening on GnuPG cryptographic agent (access for web browsers).
Apr 21 05:38:43 px1 systemd[5086]: Reached target Sockets.
Apr 21 05:38:43 px1 systemd[5086]: Reached target Basic System.
Apr 21 05:38:43 px1 systemd[5086]: Reached target Default.
Apr 21 05:38:43 px1 systemd[5086]: Startup finished in 9ms.
Apr 21 05:38:43 px1 systemd[1]: Started User Manager for UID 1000.
Apr 21 05:38:44 px1 systemd[1]: Stopping User Manager for UID 1000...
Apr 21 05:38:44 px1 systemd[5086]: Stopped target Default.
Apr 21 05:38:44 px1 systemd[5086]: Stopped target Basic System.
Apr 21 05:38:44 px1 systemd[5086]: Stopped target Timers.
Apr 21 05:38:44 px1 systemd[5086]: Stopped target Sockets.
Apr 21 05:38:44 px1 systemd[5086]: Closed GnuPG cryptographic agent and passphrase cache (restricted).
Apr 21 05:38:44 px1 systemd[5086]: Closed GnuPG cryptographic agent and passphrase cache.
Apr 21 05:38:44 px1 systemd[5086]: Stopped target Paths.
Apr 21 05:38:44 px1 systemd[5086]: Closed GnuPG cryptographic agent (access for web browsers).
Apr 21 05:38:44 px1 systemd[5086]: Closed GnuPG network certificate management daemon.
Apr 21 05:38:44 px1 systemd[5086]: Closed GnuPG cryptographic agent (ssh-agent emulation).
Apr 21 05:38:44 px1 systemd[5086]: Reached target Shutdown.
Apr 21 05:38:44 px1 systemd[5086]: Starting Exit the Session...
Apr 21 05:38:44 px1 systemd[5086]: Received SIGRTMIN+24 from PID 5282 (kill).
Apr 21 05:38:44 px1 systemd[1]: Stopped User Manager for UID 1000.
Apr 21 05:38:44 px1 systemd[1]: Removed slice User Slice of prtg.
Apr 21 05:38:50 px1 pvestatd[1461]: storage 'kvm' is not online
Apr 21 05:38:53 px1 snmpd[1009]: Connection from UDP: [192.168.0.20]:62063->[192.168.0.7]:161

pveversion:
Code:
pve-manager/5.1-51/96be5354 (running kernel: 4.13.13-6-pve)

ip a:
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
    link/ether 00:22:4d:aa:7a:7a brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:22:4d:aa:7a:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.7/24 brd 192.168.0.255 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::222:4dff:feaa:7a7a/64 scope link
       valid_lft forever preferred_lft forever
4: tap102i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UNKNOWN group default qlen 1000
    link/ether fa:25:19:7b:fc:5c brd ff:ff:ff:ff:ff:ff
5: tap201i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UNKNOWN group default qlen 1000
    link/ether d2:d4:b8:c8:5f:6a brd ff:ff:ff:ff:ff:ff

Die VM104 startet, VM102 nicht (Konfigurationsunterschiede ?)
Code:
root@px1 /etc/pve/qemu-server > cat 102.conf
agent: 1
boot: cdn
bootdisk: virtio0
cores: 2
ide2: none,media=cdrom
memory: 2048
name: web1
net0: virtio=2A:20:9F:11:C7:78,bridge=vmbr0
numa: 0
ostype: l26
smbios1: uuid=154d12d2-abdb-4e97-9c2b-dcf8d3dd62ab
sockets: 1
startup: order=3,up=120
unused0: local:102/vm-102-disk-2.raw
virtio0: local:102/vm-102-disk-1.raw,cache=writeback,size=8G

root@px1 /etc/pve/qemu-server > cat 104.conf
agent: 1
boot: cdn
bootdisk: virtio0
cores: 2
ide1: none,media=cdrom
memory: 2048
name: dms
net0: virtio=62:34:39:31:39:61,bridge=vmbr0
numa: 0
ostype: l26
smbios1: uuid=eb543ecc-0b6b-400e-b01c-9c35969706c5
sockets: 1
startup: order=5,up=30
virtio0: local:104/vm-104-disk-1.raw,cache=writeback,size=8G
 
Last edited:
Die Information starten nicht ist nicht ganz richtig, die 3 betroffenen VMs starten, sind dann nach ca. 1 Minute wieder offline, und lassen sich auch über die Webconsole nicht mehr steuern (eingefrorern?)
Per "qm list" werden sie jedoch noch als "running" angezeigt.

Da ich kein Clusterstorage habe, habe ich von einer betroffenen VM ein Backup(nfs) gemacht, gelöscht und auf anderem Node importiert, hier startet die VM problemlos und bleibt auch Online.

Leider ist mir überhaupt nicht klar was dem 1. Node fehlt, steht ja nichts hilfreiches im Log, oder sehe ich im falschen nach ?
Wie kann ich wieder auf den alten Packet-Stand von 18.3.2018 zurück (den ich im 1. Beitrag gepostet habe) ?
 
Danke für Antwort,

der NFS Storage "kvm" selbst wird nur für iso-files verwendet (und kommt von betroffener VM), die VM-Disks liegen am lokalen Storage.
Ich kann es zwar nicht ganz nachvollziehen aber es sieht danach aus das am lokalen Storage der Platz nicht mehr aussreichte,
jedoch habe ich nie eine "no space" Meldung in den Logs gesehen. Ich hatte die 3 betroffenen VMs auf anderen Node migriert, erst bei der Rückmigration meckerte das System wegen zuwenig Platz.

Fazit: Hatte wohl wenig mit dem "apt-get update" zu tun, und der free-space des lokalen Storage wird nun überwacht. Auch den nfs für die isos wurde ausgelagert.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!