Ubuntu 22.04 VM randomly being powered off

[dwilmot75]

I currently have a PVE 8.2.2 setup where it runs a single VM (from a kubernetes cluster) that uses nearly all the resources of the host. Approximately once a week or once every other week it randomly goes down, no warring nothing in logs or tasks. Just powered off completely. Its not a HW problem bc I see this on a few different hosts in the same kubernetes cluster. I am fairly new to PVE but I am somewhat exhausting my efforts on this. Any help on this is greatly appreciated.

Below is my config for my virtual machine:


Host summary:

 

[fiona]

Hi,
please post the output of pveversion -v, qm config 100 as well as the excerpt from the system logs/journal from around the time the issue happened.

 

[dwilmot75]

proxmox-ve: 8.3.0 (running kernel: 6.8.4-2-pve)
pve-manager: 8.2.2 (running version: 8.2.2/9355359cd7afbae4)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.8: 6.8.4-2
proxmox-kernel-6.8.4-2-pve-signed: 6.8.4-2
ceph-fuse: 17.2.7-pve3
corosync: 3.1.7-pve3
criu: 3.17.1-2+deb12u1
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx11
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.1
libproxmox-backup-qemu0: 1.5.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.2.0
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.6
libpve-cluster-perl: 8.0.6
libpve-common-perl: 8.2.9
libpve-guest-common-perl: 5.1.6
libpve-http-server-perl: 5.1.0
libpve-network-perl: 0.10.0
libpve-rs-perl: 0.8.8
libpve-storage-perl: 8.3.3
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.5.0-1
proxmox-backup-client: 3.3.3-1
proxmox-backup-file-restore: 3.3.3-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.3.1
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.4
pve-cluster: 8.0.10
pve-container: 5.2.4
pve-docs: 8.3.1
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.2
pve-firewall: 5.0.5
pve-firmware: 3.14-3
pve-ha-manager: 4.0.6
pve-i18n: 3.3.3
pve-qemu-kvm: 9.0.2-5
pve-xtermjs: 5.3.0-3
qemu-server: 8.3.7
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.7-pve1

 

[dwilmot75]

Nothing in the logs seem off at all. Almost like someone issued a shutdown but nothing in the task logs.

root@q-eop-dell-09:~# qm config 100
balloon: 0
boot: order=virtio0;ide2
cicustom: vendor=local:snippets/ubuntu.yaml
cipassword: **********
ciupgrade: 0
ciuser: root
cores: 40
cpu: cputype=host
ide2: local-lvm:vm-100-cloudinit,media=cdrom
ipconfig0: ip=10.48.54.84/22,gw=10.48.55.254,ip6=2620:0103:c008:000f:0010:0048:0054:0084/64,gw6=2620:0103:c008:000f:0010:0048:0055:0254
memory: 726096
meta: creation-qemu=8.1.5,ctime=1734113566
name: q-eop-dell-09-vm1
nameserver: 66.129.233.81
net0: virtio=BC:24:11:97:45:25,bridge=vmbr0
scsihw: virtio-scsi-single
searchdomain: englab.juniper.net
smbios1: uuid=8a8ddb66-2b60-4d54-a821-4be1c9a64162
sockets: 2
sshkeys: %0Assh-rsa%20AAAAB3NzaC1yc2EAAAADAQABAAACAQCvrotv%2BaHCSX8k61%2BEGIEB41yeX5Va5hwhq2qWiXs0RyIg%2FfUqZW7nsBn%2FIzsiErIkbnCCkzGZZs2KyWVLgRlwJt1D8PGca8XNAgUlrxeUejtRfaGfRzLVK2s1pQDxHCYgDz9NmWqb7KDZDmNAxr8Gy1e7C9Sou%2BthyUYTWh2wJLWCtQXme%2F57m9qpvCFAzsITcNK%2Fw32eKLlMfllFeJltM5BUGkIL%2BH2kqD%2Bn1H5Hw6q6M85mEwjAKGfId5a49sllc1jpA%2F0u%2BNXfRNTpbRGRTs%2FUVwxFBrMD7N24mpUU%2BYUW8f3qTnBMKPS%2F65SX%2FqSeSSmezSqgq0gKRNAyCnJqtzawEGduEfpADaU%2BJEvpYqRn3L4zWjqJ%2FbTmFJu1SfRa6K4T7nKplGODYKpF7bAj29v3P62dqb%2FwNLRdRsU8QPtNhMBP0XC6Woe4gKSN%2Bz3ojZBkEk6LQyTEA9Q2BeI05zeUo2SbdKCZJ3Ll%2Fn1pjOn0FpmMJQLy%2BKkKuchyBppfxXL%2Fd%2BUom1UbxoOSUs5LOtOqLkj%2Bc4iZgdNwtlE021eyO9WHTR5X%2FLXszfAyd8ynfwH6JfCiRHe%2FIrR1ni6dXims94Lpr%2BHyhJwoincTUHXQmk9W6BPF53KrEi%2FeAkhFvkIdmQfMDipiuuzWMXzLgPdVh20Vi7OKIPlf74FgRw%3D%3D%20root%40q-eop-dell-09%0A
tablet: 0
vga: std
virtio0: eop-data0:100/vm-100-disk-0.raw,backup=0,iothread=1,replicate=0,size=500G
virtio1: eop-data1:100/vm-100-disk-0.raw,backup=0,iothread=1,replicate=0,size=50G
vmgenid: ba84428b-6d34-468c-83b0-bdb373770f01

 

[fiona]

Please share the full system logs from around the time the issue happened. The logs from within the VM would also be interesting.