iscsi+vlm storage,should i add it before or after cluster creation?

[m.ardito]

I'm learning how to setup and manage a pve cluster, in a test environment (so i don't really care about loosing any vm data, btw)

After solving some sync issues, i created my first cluster, but make some mistakes. I am now about to restart but need to understand exactly how should i setup this: in order to understand what i have done, the short story, from the start:
* had a single pve1 server with several vm, both local and lvm/iscsi (nas) both running
* installed a new empty server pve2
* created a cluster on pve2 and then joined the cluster with pve1

i've learned (too late, because i've read too fast the wiki) that this is wrong, a cluster should be done the other way, because <<Please note, the node should not have any VM´s. (If yes you will get conflicts with identical VMID´s...>>

so i tried to repair:
* removed pve1 from the cluster
* removed the cluster on pve2 (wiki explains how)
* created a new cluster on pve1 with several vm, both local and lvm/iscsi (nas) both still running
* joined the cluster with pve2

then, i started migrating the local storage vm from pve1 to pve2 and back: easily.

then i tried to migrate the lvm/isci storage vm from pve1 to pve2 but the manager failed complaining that the storage was not present...but that machine was still running (!) so that storage should have been there... puzzling...

so i
* removed pve2 from cluster
* removed the cluster from pve1
* rebooted pve1

and when starting that lvm/scsi vm got a sudden stop, and the log said
/usr/sbin/qm start 112
storage 'vm_iscsi' does not exists
VM 112 start failed -

* i checked on the iscsi nas and noted that pve2 (!?!) was still connected to the target
* made a manual iscsi logout from pve2 and then rebooted pve2
* then no server was connected anymore to that iscsi target


Now, i want to create from scratch another lvm/iscsi storage but should i
1) create it on single server that will become cluster master, and then any other member joining afterwards will see it?
or
2) create the cluster master first and then create the storage on it? And before or after joining othern servers to the cluster?

Sorry for being so newbie
Thanks to anyone patient to read and/or willing to help...

Marco

 

[dietmar]

Now, i want to create from scratch another lvm/iscsi storage but should i
1) create it on single server that will become cluster master, and then any other member joining afterwards will see it?
or
2) create the cluster master first and then create the storage on it? And before or after joining othern servers to the cluster?

You can only create new storage on the cluster master. The storage configuration file (/etc/pve/storage.cfg) is then synced to all cluster nodes (periodically, each minute).

 

[m.ardito]

I have still a thing not clear, pardon me, you said:
<<You can only create new storage on the cluster master>>
i imagine two cases:
* a single pve server that will later become the cluster master, add the iscsi storage on the server via web gui before creating cluster, then create the cluster and when other member join, will replicate the same storage
* an already created cluster with master and members, add the iscsi storage on the cluster master via web gui, then every minute all members will sync /etc/pve/storage.cfg

=> are both path supported, or just the second one?
=> and:
when i have a cluster with master and members, each node has its own web gui. what are the web gui/cli operations/commands I should do always only on the cluster master and _never_ on the slave nodes (even if their web gui allows me to do them), particularly those that could break the cluster consistency?

Thanks Dietmar for you patience & support, as you said in ohter posts, pve just seems simple but it is really complex, so i (and, think, others) need to understand howto make (and possibly not to break) a good setup
Marco.

 

[dietmar]

* a single pve server that will later become the cluster master, add the iscsi storage on the server via web gui before creating cluster, then create the cluster and when other member join, will replicate the same storage
* an already created cluster with master and members, add the iscsi storage on the cluster master via web gui, then every minute all members will sync /etc/pve/storage.cfg

=> are both path supported, or just the second one?

yes.

=> and:
when i have a cluster with master and members, each node has its own web gui. what are the web gui/cli operations/commands I should do always only on the cluster master and _never_ on the slave nodes (even if their web gui allows me to do them), particularly those that could break the cluster consistency?

The web interface is 'read-only' on the nodes for anything that can break cluster consistency. So most configuration need to be done on the master. Only some items in Configuration/System (Network, DNS) are node specific.