[SOLVED] Multi node cluster, local ZFS per node

May 3, 2021
100
12
18
Hi,

What is the correct way to setup local ZFS Pools on each node? I am not looking to share them/ha across nodes, just want to use a local pool on each node for VM's, and used my shared storage for HA (NFS) across the pool.

I have the cluster setup, and when I add a zpool to a node, it looks good, I add a new zpool to another node in the cluster and the other zpool disappears.

I feel like I'm missing a step since I don't want HA/shared zpools across the cluster.

Edit - Running latest commercial 7.2.7.

Thanks!
 

oguz

Proxmox Retired Staff
Retired Staff
Nov 19, 2018
5,207
693
118
hi,

I have the cluster setup, and when I add a zpool to a node, it looks good, I add a new zpool to another node in the cluster and the other zpool disappears.
where did you add/create the zpool?

could you post your /etc/pve/storage.cfg file?
 
May 3, 2021
100
12
18
Thanks, info below. The error is different this time, it won't let me create "tank" on more than one node, it gives me an error 500.

Code:
Node 1:

Gui -> Node 1 -> ZFS - Create ZFS -> TANK -> Raid10 -> Add Storage Checked

Datacetner -> Storage -> TANK -> Edit -> Thin Provision Checked

Node 2:

Gui -> Node 1 -> ZFS - Create ZFS -> TANK -> Raid10 -> Add Storage Checked ->
Error Storage ID Tank already defined (500).

Code:
root@node1:~# cat /etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content vztmpl,iso,backup

zfspool: local-zfs
        pool rpool/data
        content rootdir,images
        sparse 1

zfspool: tank
        pool tank
        content rootdir,images
        mountpoint /tank
        nodes node1
        sparse 1

root@node1:~#

Code:
root@node1:~# pveversion -v
proxmox-ve: 7.2-1 (running kernel: 5.15.35-2-pve)
pve-manager: 7.2-7 (running version: 7.2-7/d0dd0e85)
pve-kernel-5.15: 7.2-4
pve-kernel-helper: 7.2-4
pve-kernel-5.15.35-2-pve: 5.15.35-5
pve-kernel-5.15.30-2-pve: 5.15.30-3
ceph-fuse: 15.2.16-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-3
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-2
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-3
libpve-storage-perl: 7.2-5
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.12-1
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.3-1
proxmox-backup-file-restore: 2.2.3-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-1
pve-container: 4.2-1
pve-docs: 7.2-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.4-2
pve-ha-manager: 3.3-4
pve-i18n: 2.7-2
pve-qemu-kvm: 6.2.0-11
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.4-pve1
root@node1:~#

I assume I need unique zpool names per node if I am not clustering them?
 
Last edited:

oguz

Proxmox Retired Staff
Retired Staff
Nov 19, 2018
5,207
693
118
it won't let me create "tank" on more than one node, it gives me an error 500.
in that case you should use a different name for the pool, or you can activate the storage for the other node if you want to use the same name
 
  • Like
Reactions: tjk
May 3, 2021
100
12
18
Thanks, unique names per node worked. I assume if I want to cluster/share/ha the zpool's and use replication etc I need to use the same zpool name on each node in the cluster?
 

aaron

Proxmox Staff Member
Staff member
Jun 3, 2019
3,061
512
118
As a follow up especially regarding HA: the pools need to have the same name.

If you create the pool via the GUI on the first node, you can leave the "Add Storage" checkbox enabled. On the second node, you will need to disable that checkbox for now and then adjust the nodes list for that storage when you edit it under Datacenter->Storage.

To remedy that now, you can remove the second storage from the Datacenter->Storage config and rename the pool on the second node. To do so, first export it
Code:
zpool export <name>
and then import it:
Code:
zpool import <name> <new name>
Then check the Proxmox VE Storage config and make sure that the node is also selected for that storage. Then it should show up for both and things like replication and HA should work.

I just sent a patch to make this easier in the future :)
 
  • Like
Reactions: Neobin and tjk
May 3, 2021
100
12
18
Thank you both! Not sure what the initial issue was, but adding the zpools for local use (non clustered, non HA) worked once I gave each pool a unique name. Thank you!!!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!