full-upgrade given strange situation

[achekalin]

I tried to update Proxmox 2.0 to last version with aptitude full-upgrade, and finally got a lot of 'disagrees about version of symbol module_layout' and 'invalid module format' messages on boot, and a lot of modules won't load at startup. Both modules and kernel are 2.6.32-6-pve, and I simply touch nothing by myself (nor compiled modules or kernel), so it should match. But it won't

Try to fix it by reinstall pve kernel with apt-get install --reinstall ..., it finished, but the problem is still there.

I don't even know how to copy VMs (more specifically, on Proxmox 1.8-1.9 I knew where VM configs and disks are, on 2.0 I can find only disks, no config files (it stored in Proxmox DB, I delieve), so I doubt I can simple copy outside VMs, reinstall Proxmox and copy VMs back.

What should I do now? Are there any ways to get 2.6.32-6-pve kernel and modules that match each other?

P.S. Happy new year anyone, thanks for your work on new Proxmox!

[tom]

pls post the full error logs, also 'pveversion -v'

[achekalin]

Oh, sorry. Here is is:

root@prox:/# pveversion -v
pve-manager: not correctly installed (pve-manager/2.0/16283a5a)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.32: 2.0-55
pve-kernel-2.6.32-6-pve: 2.6.32-55
lvm2: 2.02.88-2pve1
clvm: 2.02.88-2pve1
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-1
libqb: 0.6.0-1
redhat-cluster-pve: 3.1.8-3
pve-cluster: not correctly installed
qemu-server: 2.0-13
pve-firmware: 1.0-14
libpve-common-perl: 1.0-11
libpve-access-control: 1.0-5
libpve-storage-perl: 2.0-9
vncterm: 1.0-2
vzctl: 3.0.29-3pve8
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-1
ksm-control-daemon: 1.1-1

in dmesg output a lot of lines like this:

hed: disagrees about version of symbol module_layout

if I do modprobe hed I see:

FATAL: Error inserting hed (/lib/modules/2.6.32-6-pve/kernel/drivers/acpi/hed.ko): Invalid module format

but if I do modinfo hed I see

filename: /lib/modules/2.6.32-6-pve/kernel/drivers/acpi/hed.ko
license: GPL
description: ACPI Hardware Error Device Driver
author: Huang Ying
srcversion: A85C55D703C116E3AA95EFD
alias: acpi*:PNP0C33:*
depends:
vermagic: 2.6.32-6-pve SMP mod_unload modversions

The kernel is (uname -a):
Linux prox 2.6.32-6-pve #1 SMP Tue Dec 6 11:06:22 CET 2011 x86_64 GNU/Linux

I mark the version infos - looks similar.

[tom]

you did not reboot, you still running the old kernel.

[achekalin]

Noop, I did reboot (at least 4 times) and everytime the picture is still the same. Grub given me the only choice of kernel to boot, so maybe the new kernel somehow won't update everything as intended.

I tried to remove the pve-headers-2.6.32-6-pve and pve-kernel-2.6.32-6-pve, and reinstall it, then rebot the system - no change.

[tom]

your 'uname -a' shows the old version. but you installed pve-kernel-2.6.32-6-pve: 2.6.32-55

uname -a should be:

root@hp4:~# pveversion -v
pve-manager: 2.0-17 (pve-manager/2.0/ab247809)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.32: 2.0-55
pve-kernel-2.6.32-6-pve: 2.6.32-55
lvm2: 2.02.88-2pve1
clvm: 2.02.88-2pve1
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-1
libqb: 0.6.0-1
redhat-cluster-pve: 3.1.8-3
pve-cluster: 1.0-17
qemu-server: 2.0-13
pve-firmware: 1.0-14
libpve-common-perl: 1.0-11
libpve-access-control: 1.0-5
libpve-storage-perl: 2.0-9
vncterm: 1.0-2
vzctl: 3.0.29-3pve8
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-1
ksm-control-daemon: 1.1-1


root@hp4:~# uname -a
Linux hp4 2.6.32-6-pve #1 SMP Mon Dec 19 10:15:23 CET 2011 x86_64 GNU/Linux
root@hp4:~#

[achekalin]

Frankly, I missed that, thanks for pointing that out!

Ok, so what can I fix the problem? I'm simply lost, please advice!

[achekalin]

When I try to install pve-cluster and pve-manager, I see error that it cannot restart pve cluster filesystem due to lack of fuse module in kernel.

Tom, please tell me how to fix it!

[tom]

If you cannot figure it out you can reinstall from scratch.

[achekalin]

If I was able to save all of my VM on it I'd reinstall right at that moment.

How can I backup VM with settings? As I've written above I don't know where Proxmox 2.0 stores per-VM settings (nics, disks etc)

What I don't understand, all versions are the same for your an my setup, so it looks like my machine should work (I believe yours is alive ), and so I didn't understand what component should I upgrade or downgrade...

[achekalin]

After I removed kernel (apt-get remove ...), download pve-kernel-2.6.32-6-pve_2.6.32-53_amd64.deb (not pve-kernel-2.6.32-6-pve_2.6.32-55_amd64.deb, which is the freshest here), the system returned into working condition.

But as I issue apt-get update && aptitude safe-update, it become unusable (modules won't start) again. Looks like it reproducible on my system.

Will try to reinstall system as soon as I can, but as for now I'd prefer to migrate to stable 1.9 banch...