OPNSense / FreeBSD & acpi(conf) issues.

[MicahB]

I am running Proxmox 6.3-3 and have a OPNSense (FreeBSD) firewall running. I noticed several versions ago that ACPI commands were being ignored by several of my VMs.

Recently, I found some spare time and discovered that while I had enabled QEMU Guest Agent in Proxmox at VM creation acpi wasn't enabled on the guest OS. I was able to apply the simple fix for those (mostly debian) VM's of setting ACPI to autostart, but the OPNSense VM is evading me.

I have verified that I can make ACPI calls from the vm shell / terminal for the S3, S4, and S5 states in the guest, but reboot & shutdown commands from Proxmox fail with either a timeout error or a failed to run vncproxy error:

Jan 02 12:17:04 moxy pvedaemon[6090]: <root@pam> starting task UPID:moxy:00005992:007BF34D:5FF0B8A0:qmreboot:101:root@pam:
Jan 02 12:17:04 moxy pvedaemon[22930]: requesting reboot of VM 101: UPID:moxy:00005992:007BF34D:5FF0B8A0:qmreboot:101:root@pam:
Jan 02 12:27:04 moxy pvedaemon[22930]: VM 101 qmp command failed - VM 101 qmp command 'guest-shutdown' failed - got timeout
Jan 02 12:27:04 moxy pvedaemon[22930]: VM quit/powerdown failed

OPNSense: 20.7.7

PVE Version:

proxmox-ve: 6.3-1 (running kernel: 5.4.78-2-pve)
pve-manager: 6.3-3 (running version: 6.3-3/eee5f901)
pve-kernel-5.4: 6.3-3
pve-kernel-helper: 6.3-3
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-5.4.65-1-pve: 5.4.65-1
pve-kernel-4.15: 5.4-19
pve-kernel-4.15.18-30-pve: 4.15.18-58
pve-kernel-4.15.17-1-pve: 4.15.17-9
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.7
libproxmox-backup-qemu0: 1.0.2-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-2
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.1-1
libpve-storage-perl: 6.3-3
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.6-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-3
pve-cluster: 6.2-1
pve-container: 3.3-2
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-7
pve-xtermjs: 4.7.0-3
qemu-server: 6.3-2
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.5-pve1

 

[janssensm]

For OPNsense you should set options ACPI Support on Yes (default) and QEMU Guest Agent on Disabled (default).
OPNsense doesn't have QEMU guest agent.
I suppose you have QEMU set to Enabled, so Proxmox will try to talk to the agent, but it doesn't respond.

 

[MicahB]

I finally managed to capture the vncproxy error:

Jan 02 14:48:50 moxy pvedaemon[15119]: <root@pam> starting task UPID:moxy:00003C18:0089D83C:5FF0DC32:vncproxy:101:root@pam:
Jan 02 14:48:50 moxy pvedaemon[15384]: starting vnc proxy UPID:moxy:00003C18:0089D83C:5FF0DC32:vncproxy:101:root@pam:
Jan 02 14:48:50 moxy qmeventd[708]: Starting cleanup for 101
Jan 02 14:48:50 moxy qmeventd[708]: Finished cleanup for 101
Jan 02 14:48:51 moxy qm[15386]: VM 101 qmp command failed - VM 101 not running
Jan 02 14:48:51 moxy pvedaemon[15384]: Failed to run vncproxy.
Jan 02 14:48:51 moxy pvedaemon[15119]: <root@pam> end task UPID:moxy:00003C18:0089D83C:5FF0DC32:vncproxy:101:root@pam: Failed to run vncproxy.

 

[MicahB]

For OPNsense you should set options ACPI Support on Yes (default) and QEMU Guest Agent on Disabled (default).
OPNsense doesn't have QEMU guest agent.
I suppose you have QEMU set to Enabled, so Proxmox will try to talk to the agent, but it doesn't respond.

TLDR; Thanks that fixed it!

I made the changes you suggested (in this case disabling QEMU) because you are correct QEMU-guest-agent has STILL not been implemented for OPNSense / FreeBSD - I believe due to the consideration that a reboot / shutdown command can be sent to the server without authenticating (repeatedly)?

I changed my vm processor type to qemu64 at the mention that it resolved some issues for TrueNAS (also built on FreeBSD), after a reboot I was still not getting a reaction from the VM when I requested a reboot from ProxMox; however, after initiating the reboot from the vm it began responding to the ACPI requests from ProxMox without issue!

Side benefit: my OPNSense vm is booting, rebooting, and shutting down much faster now?

Thanks for your help!

edit(s): responded -> responding, TLDR

 

[janssensm]

Great that the suggestion worked. I think the agent setting was the main cause for your acpi requests not working.
With type qemu64 you get a limited set of cpu features presented to the guest.
You now should be able to select the processor type you really want, for example type "host" so you get the most performance out of your guest.
(I assume you don't use the live migration feature with opnsense, so host type can be used)

Perhaps you should check your other vm's too with this info, so if they really have a qemu guest agent installed and running while having the agent setting enabled in their options.

 

[MicahB]

Great that the suggestion worked. I think the agent setting was the main cause for your acpi requests not working.
With type qemu64 you get a limited set of cpu features presented to the guest.
You now should be able to select the processor type you really want, for example type "host" so you get the most performance out of your guest.
(I assume you don't use the live migration feature with opnsense, so host type can be used)

Perhaps you should check your other vm's too with this info, so if they really have a qemu guest agent installed and running while having the agent setting enabled in their options.

Ahh, thanks for the extra note, will do!

 

[MicahB]

Now, how do I close this?

 

[janssensm]

As thread starter you can mark the thread as 'SOLVED' yourself -> klick on the 3 dots '...' above your first post-> Edit Thread-> Prefix -> 'SOLVED'