VM shots down and starts on its own

[Teberon]

I am currently facing an issue after I created a VM that should be used to test OPNSense. The VM boots and while the installation process happens it shuts down and boot up again allowing me to start all over...
cat /var/log/syslog | grep oom - doesn't have any output (att this moment only 16GB out of 64Gb used)

cat /var/log/syslog shows below output

2024-03-24T09:07:15.273234+01:00 pve01 pvedaemon[2013]: starting 1 worker(s)
2024-03-24T09:07:15.276801+01:00 pve01 pvedaemon[2013]: worker 1461510 started
2024-03-24T09:11:18.948317+01:00 pve01 qm[1462466]: <root@pam> starting task UPID:pve01:001650D7:14D8B44C:65FFE026:qmstop:100:root@pam:
2024-03-24T09:11:18.950153+01:00 pve01 qm[1462487]: stop VM 100: UPID:pve01:001650D7:14D8B44C:65FFE026:qmstop:100:root@pam:
2024-03-24T09:11:19.439349+01:00 pve01 kernel: [3497485.336407] tap100i0: left allmulticast mode
2024-03-24T09:11:19.439365+01:00 pve01 kernel: [3497485.336435] fwbr100i0: port 2(tap100i0) entered disabled state
2024-03-24T09:11:19.471336+01:00 pve01 kernel: [3497485.368416] fwbr100i0: port 1(fwln100i0) entered disabled state
2024-03-24T09:11:19.471344+01:00 pve01 kernel: [3497485.368495] vmbr0: port 3(fwpr100p0) entered disabled state
2024-03-24T09:11:19.471347+01:00 pve01 kernel: [3497485.369848] fwln100i0 (unregistering): left allmulticast mode
2024-03-24T09:11:19.471348+01:00 pve01 kernel: [3497485.369853] fwln100i0 (unregistering): left promiscuous mode
2024-03-24T09:11:19.471349+01:00 pve01 kernel: [3497485.369857] fwbr100i0: port 1(fwln100i0) entered disabled state
2024-03-24T09:11:19.511311+01:00 pve01 kernel: [3497485.408620] fwpr100p0 (unregistering): left allmulticast mode
2024-03-24T09:11:19.511323+01:00 pve01 kernel: [3497485.408626] fwpr100p0 (unregistering): left promiscuous mode
2024-03-24T09:11:19.511324+01:00 pve01 kernel: [3497485.408628] vmbr0: port 3(fwpr100p0) entered disabled state
2024-03-24T09:11:20.167339+01:00 pve01 kernel: [3497486.064859] tap100i1: left allmulticast mode
2024-03-24T09:11:20.167357+01:00 pve01 kernel: [3497486.064885] fwbr100i1: port 2(tap100i1) entered disabled state
2024-03-24T09:11:20.199342+01:00 pve01 kernel: [3497486.096481] fwbr100i1: port 1(fwln100i1) entered disabled state
2024-03-24T09:11:20.199352+01:00 pve01 kernel: [3497486.096563] vmbr0: port 4(fwpr100p1) entered disabled state
2024-03-24T09:11:20.199353+01:00 pve01 kernel: [3497486.097582] fwln100i1 (unregistering): left allmulticast mode
2024-03-24T09:11:20.199354+01:00 pve01 kernel: [3497486.097587] fwln100i1 (unregistering): left promiscuous mode
2024-03-24T09:11:20.199355+01:00 pve01 kernel: [3497486.097589] fwbr100i1: port 1(fwln100i1) entered disabled state
2024-03-24T09:11:20.223309+01:00 pve01 kernel: [3497486.120630] fwpr100p1 (unregistering): left allmulticast mode
2024-03-24T09:11:20.223320+01:00 pve01 kernel: [3497486.120635] fwpr100p1 (unregistering): left promiscuous mode
2024-03-24T09:11:20.223322+01:00 pve01 kernel: [3497486.120637] vmbr0: port 4(fwpr100p1) entered disabled state
2024-03-24T09:11:20.863343+01:00 pve01 kernel: [3497486.760631] tap100i2: left allmulticast mode
2024-03-24T09:11:20.863360+01:00 pve01 kernel: [3497486.760659] fwbr100i2: port 2(tap100i2) entered disabled state
2024-03-24T09:11:20.895344+01:00 pve01 kernel: [3497486.792678] fwbr100i2: port 1(fwln100i2) entered disabled state
2024-03-24T09:11:20.895353+01:00 pve01 kernel: [3497486.792764] vmbr0: port 5(fwpr100p2) entered disabled state
2024-03-24T09:11:20.895355+01:00 pve01 kernel: [3497486.793566] fwln100i2 (unregistering): left allmulticast mode
2024-03-24T09:11:20.895356+01:00 pve01 kernel: [3497486.793570] fwln100i2 (unregistering): left promiscuous mode
2024-03-24T09:11:20.895357+01:00 pve01 kernel: [3497486.793572] fwbr100i2: port 1(fwln100i2) entered disabled state
2024-03-24T09:11:20.935305+01:00 pve01 kernel: [3497486.832611] fwpr100p2 (unregistering): left allmulticast mode
2024-03-24T09:11:20.935315+01:00 pve01 kernel: [3497486.832616] fwpr100p2 (unregistering): left promiscuous mode
2024-03-24T09:11:20.935316+01:00 pve01 kernel: [3497486.832619] vmbr0: port 5(fwpr100p2) entered disabled state
2024-03-24T09:11:21.176240+01:00 pve01 qmeventd[1435]: read: Connection reset by peer
2024-03-24T09:11:21.177147+01:00 pve01 pvedaemon[1285427]: VM 100 qmp command failed - unable to open monitor socket
2024-03-24T09:11:21.259410+01:00 pve01 pvedaemon[1276180]: <root@pam> end task UPID:pve01:00164B93:14D833C5:65FFDEDD:vncproxy:100:root@pam: OK
2024-03-24T09:11:21.304762+01:00 pve01 systemd[1]: 100.scope: Deactivated successfully.
2024-03-24T09:11:21.304947+01:00 pve01 systemd[1]: 100.scope: Consumed 6min 39.662s CPU time.
2024-03-24T09:11:21.975600+01:00 pve01 qm[1462466]: <root@pam> end task UPID:pve01:001650D7:14D8B44C:65FFE026:qmstop:100:root@pam: OK
2024-03-24T09:11:22.572570+01:00 pve01 qmeventd[1462527]: Starting cleanup for 100
2024-03-24T09:11:22.574483+01:00 pve01 qmeventd[1462527]: Finished cleanup for 100
2024-03-24T09:11:28.420742+01:00 pve01 qm[1462546]: <root@pam> starting task UPID:pve01:00165114:14D8B7FF:65FFE030:qmstart:100:root@pam:
2024-03-24T09:11:28.421319+01:00 pve01 qm[1462548]: start VM 100: UPID:pve01:00165114:14D8B7FF:65FFE030:qmstart:100:root@pam:
2024-03-24T09:11:28.647829+01:00 pve01 systemd[1]: Started 100.scope.
2024-03-24T09:11:29.867387+01:00 pve01 kernel: [3497495.765258] tap100i0: entered promiscuous mode
2024-03-24T09:11:29.935327+01:00 pve01 kernel: [3497495.835584] vmbr0: port 3(fwpr100p0) entered blocking state
2024-03-24T09:11:29.935348+01:00 pve01 kernel: [3497495.835594] vmbr0: port 3(fwpr100p0) entered disabled state
2024-03-24T09:11:29.935350+01:00 pve01 kernel: [3497495.835619] fwpr100p0: entered allmulticast mode
2024-03-24T09:11:29.935351+01:00 pve01 kernel: [3497495.835679] fwpr100p0: entered promiscuous mode
2024-03-24T09:11:29.935353+01:00 pve01 kernel: [3497495.835729] vmbr0: port 3(fwpr100p0) entered blocking state
2024-03-24T09:11:29.935356+01:00 pve01 kernel: [3497495.835733] vmbr0: port 3(fwpr100p0) entered forwarding state
2024-03-24T09:11:30.035327+01:00 pve01 kernel: [3497495.933813] fwbr100i0: port 1(fwln100i0) entered blocking state
2024-03-24T09:11:30.035347+01:00 pve01 kernel: [3497495.933823] fwbr100i0: port 1(fwln100i0) entered disabled state
2024-03-24T09:11:30.035349+01:00 pve01 kernel: [3497495.933851] fwln100i0: entered allmulticast mode
2024-03-24T09:11:30.035351+01:00 pve01 kernel: [3497495.933910] fwln100i0: entered promiscuous mode
2024-03-24T09:11:30.035353+01:00 pve01 kernel: [3497495.933964] fwbr100i0: port 1(fwln100i0) entered blocking state
2024-03-24T09:11:30.035354+01:00 pve01 kernel: [3497495.933968] fwbr100i0: port 1(fwln100i0) entered forwarding state
2024-03-24T09:11:30.047316+01:00 pve01 kernel: [3497495.948334] fwbr100i0: port 2(tap100i0) entered blocking state
2024-03-24T09:11:30.047332+01:00 pve01 kernel: [3497495.948343] fwbr100i0: port 2(tap100i0) entered disabled state
2024-03-24T09:11:30.047338+01:00 pve01 kernel: [3497495.948382] tap100i0: entered allmulticast mode
2024-03-24T09:11:30.051309+01:00 pve01 kernel: [3497495.948483] fwbr100i0: port 2(tap100i0) entered blocking state
2024-03-24T09:11:30.051320+01:00 pve01 kernel: [3497495.948488] fwbr100i0: port 2(tap100i0) entered forwarding state
2024-03-24T09:11:31.155314+01:00 pve01 kernel: [3497497.052908] tap100i1: entered promiscuous mode
2024-03-24T09:11:31.219470+01:00 pve01 kernel: [3497497.118159] vmbr0: port 4(fwpr100p1) entered blocking state
2024-03-24T09:11:31.219488+01:00 pve01 kernel: [3497497.118166] vmbr0: port 4(fwpr100p1) entered disabled state
2024-03-24T09:11:31.219489+01:00 pve01 kernel: [3497497.118187] fwpr100p1: entered allmulticast mode
2024-03-24T09:11:31.219489+01:00 pve01 kernel: [3497497.118246] fwpr100p1: entered promiscuous mode
2024-03-24T09:11:31.219491+01:00 pve01 kernel: [3497497.118573] vmbr0: port 4(fwpr100p1) entered blocking state
2024-03-24T09:11:31.219492+01:00 pve01 kernel: [3497497.118577] vmbr0: port 4(fwpr100p1) entered forwarding state
2024-03-24T09:11:31.315313+01:00 pve01 kernel: [3497497.212358] fwbr100i1: port 1(fwln100i1) entered blocking state
2024-03-24T09:11:31.315328+01:00 pve01 kernel: [3497497.212375] fwbr100i1: port 1(fwln100i1) entered disabled state
2024-03-24T09:11:31.315329+01:00 pve01 kernel: [3497497.212405] fwln100i1: entered allmulticast mode
2024-03-24T09:11:31.315330+01:00 pve01 kernel: [3497497.212462] fwln100i1: entered promiscuous mode
2024-03-24T09:11:31.315331+01:00 pve01 kernel: [3497497.212511] fwbr100i1: port 1(fwln100i1) entered blocking state
2024-03-24T09:11:31.315332+01:00 pve01 kernel: [3497497.212515] fwbr100i1: port 1(fwln100i1) entered forwarding state
2024-03-24T09:11:31.327321+01:00 pve01 kernel: [3497497.225591] fwbr100i1: port 2(tap100i1) entered blocking state
2024-03-24T09:11:31.327336+01:00 pve01 kernel: [3497497.225597] fwbr100i1: port 2(tap100i1) entered disabled state
2024-03-24T09:11:31.327337+01:00 pve01 kernel: [3497497.225617] tap100i1: entered allmulticast mode
2024-03-24T09:11:31.327337+01:00 pve01 kernel: [3497497.225700] fwbr100i1: port 2(tap100i1) entered blocking state
2024-03-24T09:11:31.327338+01:00 pve01 kernel: [3497497.225703] fwbr100i1: port 2(tap100i1) entered forwarding state
2024-03-24T09:11:32.427315+01:00 pve01 kernel: [3497498.325130] tap100i2: entered promiscuous mode
2024-03-24T09:11:32.495317+01:00 pve01 kernel: [3497498.395358] vmbr0: port 5(fwpr100p2) entered blocking state
2024-03-24T09:11:32.495336+01:00 pve01 kernel: [3497498.395367] vmbr0: port 5(fwpr100p2) entered disabled state
2024-03-24T09:11:32.495339+01:00 pve01 kernel: [3497498.395393] fwpr100p2: entered allmulticast mode
2024-03-24T09:11:32.495341+01:00 pve01 kernel: [3497498.395454] fwpr100p2: entered promiscuous mode
2024-03-24T09:11:32.495343+01:00 pve01 kernel: [3497498.396090] vmbr0: port 5(fwpr100p2) entered blocking state
2024-03-24T09:11:32.495345+01:00 pve01 kernel: [3497498.396094] vmbr0: port 5(fwpr100p2) entered forwarding state
2024-03-24T09:11:32.591316+01:00 pve01 kernel: [3497498.489968] fwbr100i2: port 1(fwln100i2) entered blocking state
2024-03-24T09:11:32.591334+01:00 pve01 kernel: [3497498.489976] fwbr100i2: port 1(fwln100i2) entered disabled state
2024-03-24T09:11:32.591336+01:00 pve01 kernel: [3497498.490003] fwln100i2: entered allmulticast mode
2024-03-24T09:11:32.591337+01:00 pve01 kernel: [3497498.490060] fwln100i2: entered promiscuous mode
2024-03-24T09:11:32.591338+01:00 pve01 kernel: [3497498.490110] fwbr100i2: port 1(fwln100i2) entered blocking state
2024-03-24T09:11:32.591339+01:00 pve01 kernel: [3497498.490114] fwbr100i2: port 1(fwln100i2) entered forwarding state
2024-03-24T09:11:32.603322+01:00 pve01 kernel: [3497498.503113] fwbr100i2: port 2(tap100i2) entered blocking state
2024-03-24T09:11:32.603334+01:00 pve01 kernel: [3497498.503119] fwbr100i2: port 2(tap100i2) entered disabled state
2024-03-24T09:11:32.603336+01:00 pve01 kernel: [3497498.503138] tap100i2: entered allmulticast mode
2024-03-24T09:11:32.603337+01:00 pve01 kernel: [3497498.503225] fwbr100i2: port 2(tap100i2) entered blocking state
2024-03-24T09:11:32.603338+01:00 pve01 kernel: [3497498.503229] fwbr100i2: port 2(tap100i2) entered forwarding state
2024-03-24T09:11:32.737251+01:00 pve01 qm[1462546]: <root@pam> end task UPID:pve01:00165114:14D8B7FF:65FFE030:qmstart:100:root@pam: OK
2024-03-24T09:11:32.974116+01:00 pve01 pvedaemon[1276180]: <root@pam> starting task UPID:pve01:001651F1:14D8B9C6:65FFE034:vncproxy:100:root@pam:
2024-03-24T09:11:32.974508+01:00 pve01 pvedaemon[1462769]: starting vnc proxy UPID:pve01:001651F1:14D8B9C6:65FFE034:vncproxy:100:root@pam:

qm config 100

balloon: 0
boot: order=ide2;scsi0;net0
cores: 4
cpu: host
ide2: local:iso/OPNsense-24.1-dvd-amd64.iso,media=cdrom,size=1936730K
memory: 4096
meta: creation-qemu=8.1.2,ctime=1711228490
name: OpnSense
net0: virtio=BC:24:11:25:5E:0F,bridge=vmbr0
net1: virtio=BC:24:11:94:F1:E1,bridge=vmbr0
net2: virtio=BC:24:11:93:13:16,bridge=vmbr0
numa: 0
onboot: 1
ostype: l26
scsi0: ceph:vm-100-disk-0,discard=on,iothread=1,size=32G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=909e1a39-cb3e-4d1d-b5da-032988ca263e
sockets: 1
vmgenid: 1acb0235-4ab5-443d-adcb-2ed8ad5f9c5a

Any ideas? I had that issue with some other lxc and vmx, but was able to get around after some starts.

 

[leesteken]

2024-03-24T09:11:18.950153+01:00 pve01 qm[1462487]: stop VM 100: UPID:pve01:001650D7:14D8B44C:65FFE026:qmstop:100:root@pam:
...
2024-03-24T09:11:28.421319+01:00 pve01 qm[1462548]: start VM 100: UPID:pve01:00165114:14D8B7FF:65FFE030:qmstart:100:root@pam:

This is a normal shut down and start of VM but not normal Proxmox behavior. Just a wild guess: DId you install some third-party software that monitors your VMs and restart VM every 5 minutes, when QEMU Guest Agents is not running (like on FreeBSD), and forgot about (mentioning) that?

 

[gfngfn256]

I notice your VM disk is using ceph: ceph:vm-100-disk-0,discard=on,iothread=1,size=32G,ssd=1
IDK your ceph setup - but maybe you're overloading with all those network devices on vmbr0.
Maybe try creating a new one vmbr1? Then add those devices on it.

 

[Teberon]

This is a normal shut down and start of VM but not normal Proxmox behavior. Just a wild guess: DId you install some third-party software that monitors your VMs and restart VM every 5 minutes, when QEMU Guest Agents is not running (like on FreeBSD), and forgot about (mentioning) that?

Yep, it seems at some point I have installed Monitor-All from tteck Proxmox helper scripts which has caused it. I have uninstalled it and works fine. I should better document my changes...

 

[leesteken]

Yep, it seems at some point I have installed Monitor-All from tteck Proxmox helper scripts which has caused it. I have uninstalled it and works fine. I should better document my changes...

Please ask tteck to document and warn about this more clearly, as each time this happens they are wasting effort from volunteers here (since it behaves as intended by tteck).