pmxcfs spamming into /var/log/syslog

qXt69WEV2a7fgbET · Jul 26, 2022

Hello Guys,

is it normal that so much entries are logged into /var/log/syslog?;

Code:

Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:17 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:18 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:19 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:20 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:21 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:21 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:21 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:21 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:21 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:21 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:21 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:21 xxxx-xx pmxcfs[3759029]: [status] notice: received log
Jul 25 23:40:21 xxxx-xx pmxcfs[3759029]: [status] notice: received log

i replaced my hostname with xxxx-xx.

I believe that i will not need that information and rather would like to disable it.

pveversion --verbose:

Code:

proxmox-ve: 7.2-1 (running kernel: 5.13.19-6-pve)
pve-manager: 7.2-3 (running version: 7.2-3/c743d6c1)
pve-kernel-5.15: 7.2-3
pve-kernel-helper: 7.2-3
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.35-1-pve: 5.15.35-3
pve-kernel-5.13.19-6-pve: 5.13.19-15
ceph: 16.2.7
ceph-fuse: 16.2.7
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksmtuned: 4.20150326
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-8
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-6
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.2-2
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.12-1
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.1.8-1
proxmox-backup-file-restore: 2.1.8-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-10
pve-cluster: 7.2-1
pve-container: 4.2-1
pve-docs: 7.2-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.4-2
pve-ha-manager: 3.3-4
pve-i18n: 2.7-1
pve-qemu-kvm: 6.2.0-6
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-2
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.4-pve1

Regards,
NJ

shrdlicka · Jul 26, 2022

Hi,
you are right, they are not very helpful. It looks like those messages might not needed anymore:

https://git.proxmox.com/?p=pve-clus...b0214b610e03a3c9bededfb884f7f73818faaf1#l1671

Can I ask how many nodes exist in your cluster, because this are a lot more messages than I'm seeing in my test cluster?

qXt69WEV2a7fgbET · Jul 26, 2022

Hello,

thank you for your fast response.

There are 8 nodes in my cluster, a few more have already been added and also removed.
I don't know if it matters, but we request a lot from the proxmox apis in our cluster.

pvecm status:

Code:

Cluster information
-------------------
Name:             xxxx-cluster
Config Version:   43
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Tue Jul 26 10:26:59 2022
Quorum provider:  corosync_votequorum
Nodes:            8
Node ID:          0x00000007
Ring ID:          1.3483
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   8
Highest expected: 8
Total votes:      8
Quorum:           5
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 10.254.0.10
0x00000002          1 10.254.0.11
0x00000003          1 10.254.0.4
0x00000004          1 10.254.0.8
0x00000005          1 10.254.0.14
0x00000006          1 10.254.0.7
0x00000007          1 10.254.0.12 (local)
0x00000008          1 10.254.0.13

Regards,
NJ

shrdlicka · Jul 26, 2022

I have to admit I'm not that familiar with the C code

. But what I gather is that you can take a look into "/etc/pve/.clusterlog" to see what messages are in there. At least some "received log" entries are coming messages stored in this file.

qXt69WEV2a7fgbET · Jul 26, 2022

Hello,

my guess that the requests to the proxmox api are the problem seems to be correct.
Am I right? If so, I am sorry for taking up your time.

cat /etc/pve/.clusterlog:

Code:

{
"data": [
{"uid": 1992343, "time": 1658834288, "pri": 6, "tag": "pvedaemon", "pid": 2254408, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283524, "time": 1658834288, "pri": 6, "tag": "pvedaemon", "pid": 1267421, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326866, "time": 1658834288, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283523, "time": 1658834287, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326865, "time": 1658834287, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326864, "time": 1658834287, "pri": 6, "tag": "pvedaemon", "pid": 752131, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283522, "time": 1658834287, "pri": 6, "tag": "pvedaemon", "pid": 1264282, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326863, "time": 1658834286, "pri": 6, "tag": "pvedaemon", "pid": 752131, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283521, "time": 1658834286, "pri": 6, "tag": "pvedaemon", "pid": 1264282, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326862, "time": 1658834286, "pri": 6, "tag": "pvedaemon", "pid": 750479, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 2759479, "time": 1658834286, "pri": 6, "tag": "pvedaemon", "pid": 2317273, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326861, "time": 1658834285, "pri": 6, "tag": "pvedaemon", "pid": 752131, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283520, "time": 1658834285, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326860, "time": 1658834285, "pri": 6, "tag": "pvedaemon", "pid": 752131, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283519, "time": 1658834285, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 1992342, "time": 1658834285, "pri": 6, "tag": "pvedaemon", "pid": 2254408, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 1992341, "time": 1658834285, "pri": 6, "tag": "pvedaemon", "pid": 2266152, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326859, "time": 1658834285, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283518, "time": 1658834284, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326858, "time": 1658834284, "pri": 6, "tag": "pvedaemon", "pid": 752131, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326857, "time": 1658834284, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283517, "time": 1658834284, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326856, "time": 1658834284, "pri": 6, "tag": "pvedaemon", "pid": 752131, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283516, "time": 1658834283, "pri": 6, "tag": "pvedaemon", "pid": 1264282, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326855, "time": 1658834283, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283515, "time": 1658834283, "pri": 6, "tag": "pvedaemon", "pid": 1264282, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326854, "time": 1658834282, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326853, "time": 1658834282, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283514, "time": 1658834282, "pri": 6, "tag": "pvedaemon", "pid": 1264282, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326852, "time": 1658834282, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 340578, "time": 1658834281, "pri": 6, "tag": "pvedaemon", "pid": 3790670, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283513, "time": 1658834281, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326851, "time": 1658834281, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283512, "time": 1658834281, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326850, "time": 1658834281, "pri": 6, "tag": "pvedaemon", "pid": 752131, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326849, "time": 1658834280, "pri": 6, "tag": "pvedaemon", "pid": 752131, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283511, "time": 1658834280, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326848, "time": 1658834280, "pri": 6, "tag": "pvedaemon", "pid": 750479, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283510, "time": 1658834280, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326847, "time": 1658834279, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283509, "time": 1658834279, "pri": 6, "tag": "pvedaemon", "pid": 1267421, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326846, "time": 1658834279, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326845, "time": 1658834279, "pri": 6, "tag": "pvedaemon", "pid": 750479, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283508, "time": 1658834278, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326844, "time": 1658834278, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283507, "time": 1658834278, "pri": 6, "tag": "pvedaemon", "pid": 1267421, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326843, "time": 1658834278, "pri": 6, "tag": "pvedaemon", "pid": 752131, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 6283506, "time": 1658834277, "pri": 6, "tag": "pvedaemon", "pid": 1269241, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326842, "time": 1658834277, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"},
{"uid": 7326841, "time": 1658834277, "pri": 6, "tag": "pvedaemon", "pid": 752132, "node": "xxxx-xx", "user": "root@pam", "msg": "successful auth for user 'interface@pve'"}
]
}

Regards,
NJ

shrdlicka · Jul 26, 2022

Yes, your assumption was correct

. To me, it looks like you are generating a ticket for each request, which is not necessary. You can reuse the same ticket for 2 hours, but this might make your requests a bit more complicated, I guess

.

fabian · Jul 26, 2022

or better yet, switch to using API tokens

those don't require a login/ticket generation round trip at all

https://pve.proxmox.com/pve-docs/pve-admin-guide.html#pveum_tokens

qXt69WEV2a7fgbET · Jul 26, 2022

Hello,

thank you very much.

Regards,
NJ

kyasky · Aug 8, 2024

Hello,
you could try with echo "1" >/etc/pve/.debug to enable debugging and replace those "received logs".

https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)
Regards,
K

Search

Search

pmxcfs spamming into /var/log/syslog

qXt69WEV2a7fgbET

Active Member

shrdlicka

Proxmox Retired Staff

qXt69WEV2a7fgbET

Active Member

shrdlicka

Proxmox Retired Staff

qXt69WEV2a7fgbET

Active Member

shrdlicka

Proxmox Retired Staff

fabian

Proxmox Staff Member

qXt69WEV2a7fgbET

Active Member

kyasky

New Member

We value your privacy