joining cluster messed up my local-zfs storage.cfg entry

ProjectPatatoe

New Member
Jul 15, 2023
6
0
1
I built a cluster where they were both using ext4 as rootfs. oops i actually want zfs. So i rebuilt one machine and got it removed from the cluster etc etc. I will eventually rebuild my other machine into zfs root.

On my newly rebuilt system, it was local-zfs. Once i joined it to the cluster, it changed to local-lvm. It seems the nodes line seems to be missing. I'm thinking it should be as simple as adding the nodes line to lvmthin: local-lvm right? Does anything have a sample of what the root zfs should have been? I didn't copy paste that down since... well this shouldn't have happened. Is this a bug?


/etc/pve/storage.cfg
Code:
dir: local
        path /var/lib/vz
        content iso,vztmpl,backup

lvmthin: local-lvm
        thinpool data
        vgname pve
        content rootdir,images
 
lol I realized that I have one node fully working so I just installed pve in a vm and got the config from it. Yes adding nodes to lvmthin and the zfspool fixes it. I still wanted to leave this post here as it seems like a bug. Or maybe someone will see that there is a potential issue with this further down the line?


/etc/pve/storage.cfg
Code:
dir: local
        path /var/lib/vz
        content iso,vztmpl,backup

lvmthin: local-lvm
        thinpool data
        vgname pve
        nodes pc1
        content rootdir,images

zfspool: local-zfs
        pool rpool/data
        sparse
        nodes pc2
        content images,rootdir
 
Not a bug and there is a note about it in the documentation:
All existing configuration in /etc/pve is overwritten when joining a cluster. In particular, a joining node cannot hold any guests, since guest IDs could otherwise conflict, and the node will inherit the cluster’s storage configuration. To join a node with existing guest, as a workaround, you can create a backup of each guest (using vzdump) and restore it under a different ID after joining. If the node’s storage layout differs, you will need to re-add the node’s storages, and adapt each storage’s node restriction to reflect on which nodes the storage is actually available.
https://pve.proxmox.com/wiki/Cluster_Manager#pvecm_join_node_to_cluster

Default storage configuration (/etc/pve/storage.cfg) for local, local-lvm and local-zfs:
https://pve.proxmox.com/wiki/Storage#_storage_configuration