pfsense VM internal-error

thimplicity · Jun 11, 2022

Hi,
I have switched to pfSense and I am running it on proxmox. So far everything works reltively fine (except all my config user errors in pfsense). the VM had an internal error twice and this time I had a look at /var/log/syslog/ as recommended here in the forum. This is the relevant portion I believe. The last message is me typing in my password wrong on the proxmox console:

Code:

Jun 11 09:15:50 pve systemd[1]: Finished Cleanup of Temporary Directories.
Jun 11 09:17:01 pve CRON[600853]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jun 11 09:33:42 pve chronyd[716]: Received KoD RATE from 108.61.56.35
Jun 11 10:17:01 pve CRON[609111]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jun 11 10:26:30 pve chronyd[716]: Received KoD RATE from 38.229.58.9
Jun 11 11:17:01 pve CRON[617419]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jun 11 11:22:04 pve QEMU[27851]: KVM internal error. Suberror: 3
Jun 11 11:22:04 pve QEMU[27851]: extra data[0]: 0x0000000080000b0e
Jun 11 11:22:04 pve QEMU[27851]: extra data[1]: 0x0000000000000031
Jun 11 11:22:04 pve QEMU[27851]: extra data[2]: 0x0000000000000083
Jun 11 11:22:04 pve QEMU[27851]: extra data[3]: 0x00000008002bbff8
Jun 11 11:22:04 pve QEMU[27851]: extra data[4]: 0x0000000000000000
Jun 11 11:22:04 pve QEMU[27851]: RAX=00000008002bb82c RBX=0000000000000000 RCX=00000000c0000101 RDX=00000000ffffffff
Jun 11 11:22:04 pve QEMU[27851]: RSI=0000000000000000 RDI=ffffffff8372d410 RBP=fffffe00361c08e0 RSP=ffffffff837286d0
Jun 11 11:22:04 pve QEMU[27851]: R8 =0000000000000001 R9 =0000000000000001 R10=ffffffff8372d410 R11=0000000000004000
Jun 11 11:22:04 pve QEMU[27851]: R12=ffffffff84210000 R13=000000010cce7000 R14=00000000000711b7 R15=00000000000000f4
Jun 11 11:22:04 pve QEMU[27851]: RIP=ffffffff8135b312 RFL=00010017 [----APC] CPL=0 II=0 A20=1 SMM=0 HLT=0
Jun 11 11:22:04 pve QEMU[27851]: ES =003b 0000000000000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
Jun 11 11:22:04 pve QEMU[27851]: CS =0020 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
Jun 11 11:22:04 pve QEMU[27851]: SS =0028 0000000000000000 ffffffff 00a09300 DPL=0 DS   [-WA]
Jun 11 11:22:04 pve QEMU[27851]: DS =003b 0000000000000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
Jun 11 11:22:04 pve QEMU[27851]: FS =0013 00000008002480d0 ffffffff 00c0f300 DPL=3 DS   [-WA]
Jun 11 11:22:04 pve QEMU[27851]: GS =001b ffffffff84210000 ffffffff 00c0f300 DPL=3 DS   [-WA]
Jun 11 11:22:04 pve QEMU[27851]: LDT=0000 0000000000000000 ffffffff 00c00000
Jun 11 11:22:04 pve QEMU[27851]: TR =0048 ffffffff837197a0 00002068 00008b00 DPL=0 TSS64-busy
Jun 11 11:22:04 pve QEMU[27851]: GDT=     ffffffff8371ffa0 00000067
Jun 11 11:22:04 pve QEMU[27851]: IDT=     ffffffff837187a0 00000fff
Jun 11 11:22:04 pve QEMU[27851]: CR0=80050033 CR2=ffffffff8135b312 CR3=00000008002bb82c CR4=000006e0
Jun 11 11:22:04 pve QEMU[27851]: DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
Jun 11 11:22:04 pve QEMU[27851]: DR6=00000000ffff0ff0 DR7=0000000000000400
Jun 11 11:22:04 pve QEMU[27851]: EFER=0000000000000d01
Jun 11 11:22:04 pve QEMU[27851]: Code=?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? <??> ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
Jun 11 11:33:00 pve kernel: [268516.733491] igc 0000:02:00.0 enp2s0: NIC Link is Down
Jun 11 11:33:00 pve kernel: [268516.733573] vmbr0: port 1(enp2s0) entered disabled state
Jun 11 11:33:14 pve kernel: [268531.269504] igc 0000:02:00.0 enp2s0: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Jun 11 11:33:14 pve kernel: [268531.270665] vmbr0: port 1(enp2s0) entered blocking state
Jun 11 11:33:14 pve kernel: [268531.270668] vmbr0: port 1(enp2s0) entered forwarding state
Jun 11 11:33:35 pve pvedaemon[250615]: authentication failure; rhost=::ffff:10.23.10.13 user=root@pam msg=Authentication failure

Any ideas what could cause this? It seems like pfsense just froze. Seems to start with "KVM internal error. Suberror: 3"

Thanks for some ideas.

dcsapak · Jun 13, 2022

not really a lot of info out there about this kvm error...
what is the output of 'pveversion -v' ?
can you also post the vm config? (qm config ID)

Valantur · Aug 12, 2022

dcsapak said:
not really a lot of info out there about this kvm error...
what is the output of 'pveversion -v' ?
can you also post the vm config? (qm config ID)

Hi @dcsapak , I just saw your message while I was trying to troubleshoot this same issue.
I'm running OPNSense instead of PFSense, but the issue is the same. I was hoping you could help me troubleshoot.
In case it helps, the host computer is a Mini PC Intel Celeron N5105 4xIntel I225-V 2.5G LAN.

Here are the results of the commands you asked the other user to run:

qm config:

boot: order=virtio0;ide2
cores: 4
cpu: host
ide2: none,media=cdrom
memory: 4096
meta: creation-qemu=6.2.0,ctime=1660068923
name: FW
net0: virtio=1E

9:B1

4

C:2C,bridge=vmbr1
net1: virtio=DE:0A:89:2B:E9:51,bridge=vmbr2
net2: virtio=BA:18:7D:55:4F:56,bridge=vmbr3
numa: 0
onboot: 1
ostype: other
parent: before_multiwan
scsihw: virtio-scsi-pci
smbios1: uuid=777950b0-4777-45bc-919d-b7489992a4aa
sockets: 1
startup: order=1
tablet: 0
virtio0: local-zfs:vm-101-disk-0,size=8G
vmgenid: c32f433a-d1ca-4b30-be26-23e4fd4ea444

pveversion -v:

proxmox-ve: 7.2-1 (running kernel: 5.15.39-3-pve)
pve-manager: 7.2-7 (running version: 7.2-7/d0dd0e85)
pve-kernel-5.15: 7.2-8
pve-kernel-helper: 7.2-8
pve-kernel-5.15.39-3-pve: 5.15.39-3
pve-kernel-5.15.39-1-pve: 5.15.39-1
pve-kernel-5.15.30-2-pve: 5.15.30-3
ceph-fuse: 15.2.16-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-2
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-3
libpve-storage-perl: 7.2-7
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
openvswitch-switch: 2.15.0+ds1-2+deb11u1
proxmox-backup-client: 2.2.5-1
proxmox-backup-file-restore: 2.2.5-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-2
pve-container: 4.2-2
pve-docs: 7.2-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.5-1
pve-ha-manager: 3.4.0
pve-i18n: 2.7-2
pve-qemu-kvm: 6.2.0-11
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.5-pve1

bobmc · Aug 12, 2022

would certainly appear that there is an issue with the N5105

https://forum.proxmox.com/search/5335653/?q=N5105&o=date

Neobin · Aug 13, 2022

Valantur said:
In case it helps, the host computer is a Mini PC Intel Celeron N5105 4xIntel I225-V 2.5G LAN.

https://bugzilla.proxmox.com/show_bug.cgi?id=4188

thimplicity · Aug 18, 2022

I have stabilized my box for 12 days now (knockonwood) with the help of this thread:
https://forums.servethehome.com/ind...e-quad-i225v-mini-pc-report.36699/post-345447

Valantur · Aug 23, 2022

I tried almost everything and am still experience a VM freeze every 4 hours. I have a cron job every 5 minutes that issues a stop and start, but it's a sad state of affairs.
OPNSense virtualized on Proxmox.

Neobin · Aug 23, 2022

For reference; this has become the "main" thread for the problem related to the N5105:
https://forum.proxmox.com/threads/vm-freezes-irregularly.111494

Search

Search

pfsense VM internal-error

thimplicity

Member

dcsapak

Proxmox Staff Member

Valantur

Active Member

bobmc

Renowned Member

Neobin

Distinguished Member