pvesr segfault

[jsager]

I have been chasing some segfaults that have been appearing on my pve node. A couple of them (NVidia driver and MySQL Webmin) I was able to track down and fix. However, I keep getting the following segfault related to pvesr:

pvesr[4243]: segfault at 24 ip 000055ee7c6feb25 sp 00007ffd1700b2a0 error 4 in perl[55ee7c61b000+1e6000]

I can see it has a relationship with perl, but I have not idea how to track down what the driver is. Has anyone seen this before and/or can offer suggestions on how to triage this issue? I don't get random segfaults. They are always associated with pvesr and perl.

Spoiler: My System Components

AMD Ryzen 7 1800X Processor (YD180XBCAEWOF)
GIGABYTE GA-AX370-Gaming (rev. 1.0) AM4 AMD X370 SATA 6Gb/s USB 3.1 HDMI ATX AMD Motherboard
32G of memory: Corsair Vengeance LPX 16GB (2x8GB) DDR4 DRAM 2400MHz C16 Desktop Memory Kit - Black (CMK16GX4M2A2400C16)
Intel PRO/1000 VT Quad Port Server Adapter LP PCI-E EXPI9404VT
Samsung 960 EVO Series - 250GB PCIe NVMe - M.2 Internal SSD (MZ-V6E250BW)
ADATA SU800 256GB 3D-NAND 2.5 Inch SATA III High Speed Read & Write up to 560MB/s & 520MB/s Solid State Drive (ASU800SS-256GT-C)
2 of ADATA SU800 128GB 3D-NAND 2.5 Inch SATA III High Speed up to 560MB/s Read Solid State Drive (ASU800SS-128GT-C)

Spoiler: pveversion -v

proxmox-ve: 5.1-40 (running kernel: 4.13.13-6-pve)
pve-manager: 5.1-46 (running version: 5.1-46/ae8241d4)
pve-kernel-4.13.13-6-pve: 4.13.13-40
pve-kernel-4.13.13-5-pve: 4.13.13-38
pve-kernel-4.13.13-2-pve: 4.13.13-33
corosync: 2.4.2-pve3
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-common-perl: 5.0-27
libpve-guest-common-perl: 2.0-14
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-17
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 2.1.1-2
lxcfs: 2.0.8-2
novnc-pve: 0.6-4
proxmox-widget-toolkit: 1.0-11
pve-cluster: 5.0-20
pve-container: 2.0-19
pve-docs: 5.1-16
pve-firewall: 3.0-5
pve-firmware: 2.0-3
pve-ha-manager: 2.0-5
pve-i18n: 1.0-4
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.9.1-8
pve-xtermjs: 1.0-2
qemu-server: 5.0-21
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.4-pve2~bpo9

Spoiler: pvesr.timer settings

pvesr.timer settings:
[Unit]
Description=Proxmox VE replication runner

[Timer]
AccuracySec=1
RemainAfterElapse=no

[Timer]
OnCalendar=minutely

[Install]
WantedBy=timers.target

Thanks.

 

[e100]

Have you rebooted into memtest and checked your RAM?
That would be my first suggestion.

If you cannot afford the downtime you might try installing and using memtester:

apt-get install memtester

 

[jsager]

Thanks for the reply. I had checked my ram when I put the system together, but will do so again to rule that out. The only thing that steered me away from that is that the segfault I get is always the one noted above, pvesr and perl. I will check out the memtester you noted, as well.

 

[fabian]

I'd also recommend installing and running "debsums" to ensure that all package files are uncorrupted. a segfault can also be caused by a corrupted binary on-disk, especially if the segfault always occurs in the same program/library.

 

[jsager]

Thanks for all the replies. I checked my memory and found that one stick had, in fact, become faulty. It started throwing errors the moment I started memtest. In the future, I will immediately check the memory should another segfault rear its ugly head. Thanks also for the debsums advice. I ran it and all package files appear to be okay. Hopefully, my issue has been solved with the memory. It took a few days for the segfault to appear, so we'll see if anything crops up over the next few days. Thanks again for all the assistance.