Its not a default feature to list changelogs.
Thats because you installed "apt-listchanges" or "apt-listbugs"
You can run "apt full-upgrade -y" and it will not ask you
[SOLVED] ZFS not enabled or running after update
- Thread starter Curt Hall
- Start date
Its not a default feature to list changelogs.
Thats because you installed "apt-listchanges" or "apt-listbugs"
You can run "apt full-upgrade -y" and it will not ask you
I folowed the steps in wiki exactly.
I know just enought to be dangerous
But not enought to be creative.
I had a functioning server at the end, but all my lxc configs were gone. And new setup didn't want to recognize the vm disks at all. They would attach but not boot.
Well the upgrade finally took hold. but now that host doesn't seem 100% compatible with the other ones in the cluster (all other hosts red X). and the shell window looks bizarre and is non-functional. Do I need to do the upgrade across all hosts in the cluster?
I also get this error when trying to add a new zfs on the newly upgraded proxmox5 to the cluster:
create storage failed: error during cfs-locked 'file-storage_cfg' operation: no quorum! (500)
This still because all hosts must be same version?
create storage failed: error during cfs-locked 'file-storage_cfg' operation: no quorum! (500)
This still because all hosts must be same version?
No your cluster is broken. Do not upgrade the other nodes yet.
Make sure your current host is fully working.
Please post output of "pvecm status" and also check "systemctl status" for failed count, must be 0
To know which services failed please post the output of:
systemctl list-units --state=failed
systemctl list-units --state=queued
# also check log of each service that failed
journalctl -u <service>
Your node is not part of the cluster.
You can try to add it
pvecm add <other cluster node ip> --force
root@proxmox5:~# systemctl list-units --state=failed
UNIT LOAD ACTIVE SUB DESCRIPTION
● lvm2-pvscan@253:16.service loaded failed failed LVM event activation on device 253:16
● zfs-import-cache.service loaded failed failed Import ZFS pools by cache file
LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.
2 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
root@proxmox5:~# systemctl list-units --state=queued
0 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
It seems like there is a problem with your lvm disk 253:16. Due to that zfs also fails.
Maybe its related to cephs/nautilus upgrade.
What does your storage setup look like ? Is it local, external etc. Are u running ceph ?
You will need to fix the storage first.
Once the storage is fixed, you can add it back to the cluster and then upgrade the other nodes in the same way.
No Ceph.
One ISCSI target to a Nimble SAN. (LVM)
Trying to add zfs local RAIDZ using on board JBOD HDD's, which shows in pool list:
root@proxmox5:~# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
RAIDZ 2.71T 528K 2.71T - - 0% 0% 1.00x ONLINE -
root@proxmox5:~#
but doesn't show up as a usable volume:
Hmm ok.
After creating the zpool you have to add it to the proxmox gui.
You can add the zfs storage to proxmox gui this way:
Datacenter -> Storage -> Add -> ZFS -> "ID: RAIDZ" Pool: Select your "RAIDZ" storage
However i kinda have the feeling your broken lvm mount is blocking proxmox.
Are you using encryption by any chance on lvm or zfs ? The last update broke encryption afaik.
No encryption.
If i login to 1 of the other 4 hosts and goto Datacenter -> Storage -> Add -> ZFS -> then RAIDZ does not populate only "replication" (an existing zfs)
If i login to Proxmox5 and goto Datacenter -> Storage -> Add -> ZFS -> then RAIDZ does populate but not "replication"
clearly proxmox5 is not really on the cluster.
Except the GUI says it is:
You will need to fix the storage lvm/zfs.
What does "journalctl -u lvm2-pvscan@253:16.service" output ?
Also "pvscan" "lsblk" "df -h" "zfs list"
root@proxmox5:~# journalctl -u lvm2-pvscan@253:16.service
-- Logs begin at Fri 2020-05-29 14:38:34 CDT, end at Fri 2020-05-29 15:45:20 CDT. --
-- No entries --
root@proxmox5:~# pvscan
PV /dev/sdg3 VG pve lvm2 [<184.83 GiB / <16.00 GiB free]
PV /dev/mapper/2f516411d15cedb126c9ce9008e5e60a7 VG nimblenas lvm2 [<5.00 TiB / 167.75 GiB free]
Total: 2 [5.18 TiB] / in use: 2 [5.18 TiB] / in no VG: 0 [0 ]
root@proxmox5:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 279.4G 0 disk
├─sda1 8:1 0 279.4G 0 part
└─sda9 8:9 0 8M 0 part
sdb 8:16 0 838.4G 0 disk
├─sdb1 8:17 0 838.4G 0 part
└─sdb9 8:25 0 8M 0 part
sdc 8:32 0 838.4G 0 disk
├─sdc1 8:33 0 838.4G 0 part
└─sdc9 8:41 0 8M 0 part
sdd 8:48 0 279.4G 0 disk
├─sdd1 8:49 0 279.4G 0 part
└─sdd9 8:57 0 8M 0 part
sde 8:64 0 279.4G 0 disk
├─sde1 8:65 0 279.4G 0 part
└─sde9 8:73 0 8M 0 part
sdf 8:80 0 279.4G 0 disk
├─sdf1 8:81 0 279.4G 0 part
└─sdf9 8:89 0 8M 0 part
sdg 8:96 0 185.3G 0 disk
├─sdg1 8:97 0 1007K 0 part
├─sdg2 8:98 0 512M 0 part
└─sdg3 8:99 0 184.9G 0 part
├─pve-root 253:0 0 46G 0 lvm /
├─pve-swap 253:1 0 8G 0 lvm [SWAP]
├─pve-data_tmeta 253:2 0 1.2G 0 lvm
│ └─pve-data-tpool 253:4 0 112.5G 0 lvm
│ └─pve-data 253:5 0 112.5G 0 lvm
└─pve-data_tdata 253:3 0 112.5G 0 lvm
└─pve-data-tpool 253:4 0 112.5G 0 lvm
└─pve-data 253:5 0 112.5G 0 lvm
sdh 8:112 0 5T 0 disk
└─2f516411d15cedb126c9ce9008e5e60a7 253:6 0 5T 0 mpath
sdi 8:128 0 5T 0 disk
└─2f516411d15cedb126c9ce9008e5e60a7 253:6 0 5T 0 mpath
sdj 8:144 0 5T 0 disk
└─2f516411d15cedb126c9ce9008e5e60a7 253:6 0 5T 0 mpath
sdk 8:160 0 5T 0 disk
└─2f516411d15cedb126c9ce9008e5e60a7 253:6 0 5T 0 mpath
root@proxmox5:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 63G 0 63G 0% /dev
tmpfs 13G 9.6M 13G 1% /run
/dev/mapper/pve-root 46G 12G 31G 28% /
tmpfs 63G 66M 63G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 63G 0 63G 0% /sys/fs/cgroup
RAIDZ 2.7T 128K 2.7T 1% /RAIDZ
/dev/fuse 30M 76K 30M 1% /etc/pve
tmpfs 13G 0 13G 0% /run/user/0
root@proxmox5:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
RAIDZ 564K 2.63T 96K /RAIDZ
Wow, i completely blew away the proxmox5 machine and reinstalled from scratch, it also has the ZFS not running error. exactly the error that the old box had when i first started this thread. unbelievable?! Maybe hardware bad, but how could that effect zfs system from even running?
when I try running zpool list, then '/sbin/modprobe zfs' as it suggests, I get this in the terminal:
root@proxmox5:~# zpool list
The ZFS modules are not loaded.
Try running '/sbin/modprobe zfs' as root to load them.
root@proxmox5:~# /sbin/modprobe zfs
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/5.4.41-1-pve/modules.dep.bin'
modprobe: FATAL: Module zfs not found in directory /lib/modules/5.4.41-1-pve
when I try running zpool list, then '/sbin/modprobe zfs' as it suggests, I get this in the terminal:
root@proxmox5:~# zpool list
The ZFS modules are not loaded.
Try running '/sbin/modprobe zfs' as root to load them.
root@proxmox5:~# /sbin/modprobe zfs
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/5.4.41-1-pve/modules.dep.bin'
modprobe: FATAL: Module zfs not found in directory /lib/modules/5.4.41-1-pve
Last edited:
Joining a cluster will sync a lot of config files in /etc/pve.
Can you send a screenshot of "Datacenter" -> "Storage"
Last edited:
So the cluster join syncs the storage and as you defined iscsi/lvm as shared proxmox5 tries to import it and fails.
The problem with that is that zfs has a dependency on other storage types and thus fails too.
Limit shared storages to production nodes that use it.
Double click it and then in the "Nodes:" tab select all nodes except proxmox5.
Afterwards reboot proxmox5 and zfs should work.
