Error in packages for 8.4.1

[jaffinio]

Hello friends,

This is my first post here and I need to share this finding. I have found the culprit of my Web GUI not working on any device in any browser. Simply giving me a blank page, but in default background color, which was nice.

---

It all came down to downgrading widget toolkit from proxmox-widget-toolkit_4.3.10_all.deb to proxmox-widget-toolkit_4.3.9_all.deb with dpkg. apt clean && apt update did not resolve the issue because of newer version available in the repositories.

---

Before this I had lots of errors in browsers pointing to proxlomxlib.js line 566 where some bracket was supposedly misplaced.

dpkg -V produced this output:

dpkg -V proxmox-widget-toolkit                                                
??5??????   /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js

Because of this file many other reported issues. Certs were good, pveproxy, pvemanager worked fine. There is some error in the source code for 4.3.10.

my pveversion output:

proxmox-ve: 8.4.0 (running kernel: 6.8.12-9-pve)
pve-manager: 8.4.1 (running version: 8.4.1/2a5fa54a8503f96d)
proxmox-kernel-helper: 8.1.1
proxmox-kernel-6.8.12-10-pve-signed: 6.8.12-10
proxmox-kernel-6.8: 6.8.12-10
proxmox-kernel-6.8.12-9-pve-signed: 6.8.12-9
amd64-microcode: 3.20240820.1~deb12u1
ceph-fuse: 16.2.15+ds-0+deb12u1
corosync: 3.1.9-pve1
criu: 3.17.1-2+deb12u1
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx11
intel-microcode: 3.20250512.1~deb12u1
libjs-extjs: 7.0.0-5
libknet1: 1.30-pve2
libproxmox-acme-perl: 1.6.0
libproxmox-backup-qemu0: 1.5.1
libproxmox-rs-perl: 0.3.5
libpve-access-control: 8.2.2
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.1.0
libpve-cluster-perl: 8.1.0
libpve-common-perl: 8.3.1
libpve-guest-common-perl: 5.2.2
libpve-http-server-perl: 5.2.2
libpve-network-perl: 0.11.2
libpve-rs-perl: 0.9.4
libpve-storage-perl: 8.3.6
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.6.0-2
proxmox-backup-client: 3.4.1-1
proxmox-backup-file-restore: 3.4.1-1
proxmox-firewall: 0.7.1
proxmox-kernel-helper: 8.1.1
proxmox-mail-forward: 0.3.2
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.9
pve-cluster: 8.1.0
pve-container: 5.2.6
pve-docs: 8.4.0
pve-edk2-firmware: not correctly installed
pve-esxi-import-tools: 0.7.4
pve-firewall: 5.1.1
pve-firmware: 3.15-3
pve-ha-manager: 4.0.7
pve-i18n: 3.4.2
pve-qemu-kvm: 9.2.0-5
pve-xtermjs: 5.5.0-2
qemu-server: 8.3.12
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.7-pve2

 

[cheiss]

Hi,

did you by any chance run some "post-installation script" from the internet or anything of the sorts?
These scripts are known to mess with system files and potentially introduce corruptions like this, especially the Javascript file you mentioned.

Normally, this can be fixed by cleanly re-installing the affected package using apt update && apt reinstall -y proxmox-widget-toolkit.
Notice the reinstall, not install, which instructs apt to reinstall the package fresh, even if the same version is already installed.

 

[jaffinio]

Hi,

did you by any chance run some "post-installation script" from the internet or anything of the sorts?
These scripts are known to mess with system files and potentially introduce corruptions like this, especially the Javascript file you mentioned.

Normally, this can be fixed by cleanly re-installing the affected package using apt update && apt reinstall -y proxmox-widget-toolkit.
Notice the reinstall, not install, which instructs apt to reinstall the package fresh, even if the same version is already installed.

Hi,

No I did not use any script from the internet. After one of my nodes went down (2 of 4 disks said bye at the same time) I have installed Proxmox from scratch using 8.1 ISO.
The error happened after upgrading from Web GUI. It might have been some ambiguous error. I did not catch it. I tried apt clean && apt update && apt install --reinstall with pvemanager, proxmox-widget-toolkit to no avail. Then it hit me to install from the repository. And voila. All is working now.

What is left to do is nano /etc/apt/preferences.d/proxmox-widget-toolkit

Package: proxmox-widget-toolkit
Pin: version 4.3.10
Pin-Priority: -1

apt update and call it a day.

And maybe check apt-cache policy proxmox-widget-toolkit to be sure

proxmox-widget-toolkit:
  Installed: 4.3.9
  Candidate: 4.3.9
  Version table:
     4.3.10 -1
        500 http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 Packages
 *** 4.3.9 500
        500 http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 Packages
        100 /var/lib/dpkg/status
     4.3.8 500
        500 http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 Packages
     4.3.7 500
...

Ohhh, I almost forgot to mention that I did check 4.3.10 from the web repo and it did not work.

 

[LnxBil]

dpkg -V proxmox-widget-toolkit ??5?????? /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js

files normally don't change by themselves on the disk and the 5 there suggests, that the file was edited by someone manually and is not the same anymore as it should be according to the package. The md5 checksum is off.

 

[jaffinio]

but when I install 4.3.10 from repo, unlock 4.3.10 in apt the error comes back

 

[cheiss]

Can you please provide the full outputs of
- ls -lah /etc/apt/apt.conf.d/ /etc/apt/preferences.d/
- apt reinstall -y proxmox-widget-toolkit=4.3.10
- apt list --installed | grep -E '(pmx|pve)'

 

[jaffinio]

ls -lah /etc/apt/apt.conf.d/ /etc/apt/preferences.d/

/etc/apt/apt.conf.d/:
total 48K
drwxr-xr-x  2 root root 4.0K May 26 05:07 .
drwxr-xr-x 10 root root 4.0K May 23 19:00 ..
-rw-r--r--  1 root root  399 May 25  2023 01autoremove
-rw-r--r--  1 root root  269 May  2  2023 10pveapthook
-rw-r--r--  1 root root   80 Dec 31  2022 20auto-upgrades
-rw-r--r--  1 root root  307 Mar 28  2021 20listchanges
-rw-r--r--  1 root root 7.2K Dec 31  2022 50unattended-upgrades
-rw-r--r--  1 root root  182 Jan  8  2023 70debconf
-r--r--r--  1 root root  547 May 24 06:36 76pveconf
-rw-r--r--  1 root root   23 May 26 05:07 76pveproxy
-rw-r--r--  1 root root   65 Apr 16 11:39 99proxmox-nag-patch

/etc/apt/preferences.d/:
total 12K
drwxr-xr-x  2 root root 4.0K May 25 23:18 .
drwxr-xr-x 10 root root 4.0K May 23 19:00 ..
-rw-r--r--  1 root root   69 May 25 23:18 proxmox-widget-toolkit

apt reinstall -y proxmox-widget-toolkit=4.3.10

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be upgraded:
  proxmox-widget-toolkit
1 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
Need to get 221 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 proxmox-widget-toolkit all 4.3.10 [221 kB]
Fetched 221 kB in 0s (675 kB/s)              
apt-listchanges: Reading changelogs...
(Reading database ... 57179 files and directories currently installed.)
Preparing to unpack .../proxmox-widget-toolkit_4.3.10_all.deb ...
Unpacking proxmox-widget-toolkit (4.3.10) over (4.3.9) ...
Setting up proxmox-widget-toolkit (4.3.10) ...

apt list --installed | grep -E '(pmx|pve)'

corosync/stable,now 3.1.9-pve1 amd64 [installed,automatic]            
grub-common/stable,now 2.06-13+pmx6 amd64 [installed,automatic]
grub-efi-amd64-bin/stable,now 2.06-13+pmx6 amd64 [installed,automatic]
grub-efi-amd64-signed/stable,now 1+2.06+13+pmx6 amd64 [installed,automatic]
grub-efi-amd64/stable,now 2.06-13+pmx6 amd64 [installed]    
grub2-common/stable,now 2.06-13+pmx6 amd64 [installed,automatic]
ifupdown2/stable,now 3.2.0-1+pmx11 all [installed,automatic]      
libcfg7/stable,now 3.1.9-pve1 amd64 [installed,automatic]
libcmap4/stable,now 3.1.9-pve1 amd64 [installed,automatic]
libcorosync-common4/stable,now 3.1.9-pve1 amd64 [installed,automatic]
libcpg4/stable,now 3.1.9-pve1 amd64 [installed,automatic]
libjs-qrcodejs/stable,now 1.20230525-pve1 all [installed,automatic]
libknet1/stable,now 1.30-pve2 amd64 [installed,automatic]
libnozzle1/stable,now 1.30-pve2 amd64 [installed,automatic]
libnvpair3linux/stable,now 2.2.7-pve2 amd64 [installed,automatic]
libopeniscsiusr/stable,now 2.1.8-1.pve1 amd64 [installed,automatic]
libpve-access-control/stable,now 8.2.2 all [installed,automatic]
libpve-apiclient-perl/stable,now 3.3.2 all [installed,automatic]
libpve-cluster-api-perl/stable,now 8.1.0 all [installed,automatic]
libpve-cluster-perl/stable,now 8.1.0 all [installed,automatic]
libpve-common-perl/stable,now 8.3.1 all [installed,automatic]
libpve-guest-common-perl/stable,now 5.2.2 all [installed,automatic]
libpve-http-server-perl/stable,now 5.2.2 all [installed,automatic]
libpve-network-api-perl/stable,now 0.11.2 all [installed,automatic]
libpve-network-perl/stable,now 0.11.2 all [installed,automatic]
libpve-notify-perl/stable,now 8.1.0 all [installed,automatic]
libpve-rs-perl/stable,now 0.9.4 amd64 [installed,automatic]
libpve-storage-perl/stable,now 8.3.6 all [installed,automatic]
libpve-u2f-server-perl/stable,now 1.2.0 amd64 [installed,automatic]
libquorum5/stable,now 3.1.9-pve1 amd64 [installed,automatic]
libtpms0/stable,now 0.9.6+pve1 amd64 [installed,automatic]
libuutil3linux/stable,now 2.2.7-pve2 amd64 [installed,automatic]
libvotequorum8/stable,now 3.1.9-pve1 amd64 [installed,automatic]
libzfs4linux/stable,now 2.2.7-pve2 amd64 [installed,automatic]
libzpool5linux/stable,now 2.2.7-pve2 amd64 [installed,automatic]
lxc-pve/stable,now 6.0.0-1 amd64 [installed,automatic]
lxcfs/stable,now 6.0.0-pve2 amd64 [installed,automatic]
novnc-pve/stable,now 1.6.0-2 all [installed,automatic]
open-iscsi/stable,now 2.1.8-1.pve1 amd64 [installed]
proxmox-kernel-6.8.12-10-pve-signed/stable,now 6.8.12-10 amd64 [installed,automatic]
proxmox-kernel-6.8.12-9-pve-signed/stable,now 6.8.12-9 amd64 [installed,automatic]
pve-cluster/stable,now 8.1.0 amd64 [installed,automatic]
pve-container/stable,now 5.2.6 all [installed,automatic]
pve-docs/stable,now 8.4.0 all [installed,automatic]
pve-edk2-firmware-legacy/stable,now 4.2025.02-3 all [installed,automatic]
pve-edk2-firmware-ovmf/stable,now 4.2025.02-3 all [installed,automatic]
pve-esxi-import-tools/stable,now 0.7.4 amd64 [installed,automatic]
pve-firewall/stable,now 5.1.1 amd64 [installed,automatic]
pve-firmware/stable,now 3.15-3 all [installed,upgradable to: 3.15-4]
pve-ha-manager/stable,now 4.0.7 amd64 [installed,automatic]
pve-i18n/stable,now 3.4.2 all [installed,upgradable to: 3.4.4]
pve-lxc-syscalld/stable,now 1.3.0 amd64 [installed,automatic]
pve-manager/stable,now 8.4.1 all [installed,automatic]
pve-nvidia-vgpu-helper/stable,now 0.2.0 all [installed,automatic]
pve-qemu-kvm/stable,now 9.2.0-5 amd64 [installed,automatic]
pve-xtermjs/stable,now 5.5.0-2 all [installed,automatic]
shim-helpers-amd64-signed/stable,now 1+15.8+1+pmx1 amd64 [installed,automatic]
shim-signed-common/stable,now 1.44+pmx1+15.8-1+pmx1 all [installed,automatic]
shim-signed/stable,now 1.44+pmx1+15.8-1+pmx1 amd64 [installed,automatic]
shim-unsigned/stable,now 15.8-1+pmx1 amd64 [installed,automatic]
smartmontools/stable,now 7.3-pve1 amd64 [installed,automatic]
swtpm-libs/stable,now 0.8.0+pve1 amd64 [installed,automatic]
swtpm-tools/stable,now 0.8.0+pve1 amd64 [installed,automatic]
swtpm/stable,now 0.8.0+pve1 amd64 [installed,automatic]
virtiofsd/stable,now 1.10.1-1~bpo12+pve1 amd64 [installed,automatic]
zfs-initramfs/stable,now 2.2.7-pve2 all [installed]
zfs-zed/stable,now 2.2.7-pve2 amd64 [installed,automatic]
zfsutils-linux/stable,now 2.2.7-pve2 amd64 [installed,automatic]

aaaaaand after that WebGUI does not work....

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
    <title>prox - Proxmox Virtual Environment</title>
    <link rel="icon" sizes="128x128" href="/pve2/images/logo-128.png" />
    <link rel="apple-touch-icon" sizes="128x128" href="/pve2/images/logo-128.png" />
    <link rel="stylesheet" type="text/css" href="/pve2/ext6/theme-crisp/resources/theme-crisp-all.css?ver=7.0.0" />
    <link rel="stylesheet" type="text/css" href="/pve2/ext6/crisp/resources/charts-all.css?ver=7.0.0" />
    <link rel="stylesheet" type="text/css" href="/pve2/fa/css/font-awesome.css" />
    <link rel="stylesheet" type="text/css" href="/pve2/font-logos/css/font-logos.css" />
    <link rel="stylesheet" type="text/css" href="/pve2/css/ext6-pve.css?ver=8.4.1" />
    <link rel="stylesheet" type="text/css" href="/pwt/css/ext6-pmx.css?ver=v4.3.10-t1744126616" />
    <link rel="stylesheet" type="text/css" media="(prefers-color-scheme: dark)" href="/pwt/themes/theme-proxmox-dark.css?ver=v4.3.10-t1744126616" />
   
    <script type='text/javascript'>function gettext(buf) { return buf; }</script>
   
    <script type="text/javascript" src="/pve2/ext6/ext-all.js?ver=7.0.0"></script>
    <script type="text/javascript" src="/pve2/ext6/charts.js?ver=7.0.0"></script>
   
    <script type="text/javascript" src="/pve2/js/u2f-api.js"></script>
    <script type="text/javascript" src="/qrcode.min.js"></script>
    <script type="text/javascript">
    Proxmox = {
    Setup: { auth_cookie_name: 'PVEAuthCookie' },
    defaultLang: 'en',
    NodeName: 'prox',
    UserName: '',
    CSRFPreventionToken: 'null',
    ConsentText: ''
    };
    </script>
    <script type="text/javascript" src="/proxmoxlib.js?ver=v4.3.10-t1744126616"></script>
    <script type="text/javascript" src="/pve2/js/pvemanagerlib.js?ver=8.4.1"></script>
    <script type="text/javascript" src="/pve2/ext6/locale/locale-en.js?ver=7.0.0"></script>

    <script type="text/javascript">
    if (typeof(PVE) === 'undefined') PVE = {};
    Ext.History.fieldid = 'x-history-field';
    Ext.onReady(function() { Ext.create('PVE.StdWorkspace');});
    </script>

  </head>
  <body>
    <!-- Fields required for history management -->
    <form id="history-form" class="x-hidden">
    <input type="hidden" id="x-history-field"/>
    </form>
  </body>
</html>

[EDIT]
after downgrade GUI came back

 

[cheiss]

/etc/apt/apt.conf.d/:
-rw-r--r-- 1 root root 65 Apr 16 11:39 99proxmox-nag-patch

That's the problem. So you probably did indeed run some script?

You can try removing the file and then running apt update && apt reintall proxmox-widget-toolkit again. This should fix this, but as it is some third-party modification, other system files may also been modified.