Freeze on pfSense VM running in PVE 9

T

tizbac

New Member
Dec 4, 2023
8
20
3
Hi, since i upgraded my home lab PVE from 8 to 9 i started experiencing freeze of my pfSense 2.8.0 ( FreeBSD 15 ) VM.
It's not guest OS freeze, and neither host, just qemu becoming unresponsive, even noVNC console / monitor won't work ( timeout )
It takes a lot to happen, now i installed gdb to try get a backtrace next time it happens...

Below is qemu-server config

Code: 
boot: order=scsi0
cores: 4
ide2: none,media=cdrom
memory: 2048
meta: creation-qemu=7.2.0,ctime=1682459789
name: PFSense
net0: virtio=52:54:00:ca:2e:15,bridge=vmbr1
net1: virtio=52:54:00:52:b1:9b,bridge=vmbr0
net2: virtio=9A:BD:C1:04:FD:4A,bridge=vmbr2,firewall=1
net3: virtio=D2:B3:8A:71:C6:F0,bridge=vmbr3,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: local:104/vm-104-disk-0.qcow2,cache=writeback,discard=on,iothread=1,size=30G
scsihw: virtio-scsi-single
sockets: 1
vmgenid: 8d4b90d4-2cae-4441-a7a0-f906cb425aa1

it happened for sure in pve 9.0.5 , i also upgraded to 9.0.6 today
Edit: CPU usage is 0% , suggesting deadlock
 
Last edited:
Hello,

Try to:
- edit NICs and switch NIC model use e1000 or vmxnet3.
- switch from q35 to pc or vice versa
- change CPU type
 
Just to let you know that Ihave the same issue.
My pfSense instance is completely offline and I need to stop it using CLI "qm stop" before I can use the GUI again to start the VM.
@tizbac - any news?
 
Hi,
what does qm status <ID> --verbose say while the VM is in this state (replacing <ID> with the actual number)? Please also share the system journal from around the time the issue happens.

To obtain a QEMU stack trace you can run apt install pve-qemu-kvm-dbgsym gdb. And the next time the VM is stuck, run
gdb --batch --ex 't a a bt' -p $(cat /var/run/qemu-server/123.pid) > /tmp/stuck-vm.txt (replacing 123 with the correct ID).

@UweV please also share the VM configuration qm config <ID>
 
Hello people of the forum.
I have the exact same problem but I don't think it is related to the e1000 Driver problem that you referenced here. My VM uses virtIO nics.
My PFSense VM just freezes (probably starting to freeze when the backup starts)
Everything else works just fine, I can use the webinterface no problems whatsoever and I can also kill the VM using the webintierface.
I'm on proxmox 9.0.10 and the issues started for me just about 3 days ago and I have no clue what the cause could be because it is also inconsistent.
On friday it froze (probably during the backup operation) yesterday it did not, and today it froze again.

Viewing my system log I can tell that the vm froze like 4 hours into the backup and the log starts regularly reporting
Oct 19 12:35:12 pve pvestatd[3003]: VM 101 qmp command failed - VM 101 qmp command 'query-proxmox-support' failed - unable to connect to VM 101 qmp socket - timeout after 51 retries
Normally the PFSense VM takes 4 minutes to back up (also unusually long compared to my other vms which only take like less than 40 seconds but still fine I guess...)
 
Last edited:
I have exactly the same issue. It's probably not related to E1000e known issue.
pfSense VM start freezing during backup with PBS.
Stop and start of the VM is required to recover.

qm config <ID>:
agent: 0
boot: order=ide2;scsi0
cores: 1
cpu: x86-64-v2-AES
ide2: none,media=cdrom
memory: 1024
meta: creation-qemu=8.1.5,ctime=1712415404
name: pfSense1
net0: virtio=BC:24:11:F9:D7:7E,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: ZFS1:vm-503-disk-0,iothread=1,size=10G
scsihw: virtio-scsi-single
smbios1: uuid=93811e06-87a1-491d-afd1-3e7577adca4f
sockets: 1
tags: 2;prod
vmgenid: 05090f22-7b0e-45b9-9c9e-2e851ada7f3a
 
For those running into the issue after backup, please see: https://bugzilla.proxmox.com/show_bug.cgi?id=6810

It's not necessary to provide the stack trace with GDB anymore, because the reporter of the bug already did. That trace helped a lot to find the cause of the issue. You can still check it to see if it's the same issue though.
 
  • Like
Reactions: Johannes S
You must log in or register to reply here.
Top Bottom