[SOLVED] help with cluster storage

[rusquad]

hello, it's me again. i defeated fc adapters and connected 2 servers to one storage with one shared drive, thanks everyone. now via shared lvm.
I connected them with another server in a cluster, so I have 2 servers with shared storage fc lvm + 1 server with local storage.

I wanted to clone my existing machines on the 3rd server to servers with shared storage, but I did not find a way to do this. even during migration, there is no option to select a shared storage, only the required node and the machine are moved to the local storage.

as I understand it, the general lvm does not support backups and snapshots, and this is very necessary. why does local lvm work with backups and snapshots?
what should I do? i have seen many threads with similar questions, but the problem is that they don't have specific guides, only discussions.
maybe I shouldn't be chasing shared storage? to make each node its own? but of course I want to be able to live migration. please write your experience and advise me with a more detailed answer. any suggestions! Please!

 

[dignus]

Backups will work with PBS. Snapshots on LVM are technically possible, performance goes down so much, you don't want that.

 

[bbgeek17]

I wanted to clone my existing machines on the 3rd server to servers with shared storage, but I did not find a way to do this. even during migration

Migration is not the same as changing underlying storage. In PVE GUI speak "migration" means moving the VM compute between nodes.
To change storage you can:
- select disk and use "move" option in UI
- use QM cli. See "man qm" > "qm migrate"
- use QM "qm start <vmid> [OPTIONS]" . See "--targetstorage <string>"

the general lvm does not support backups and snapshots

I dont recall off hand about any restrictions on backups, but snapshots are indeed not supported on Thick LVM shared storage.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[rusquad]

bbgeek17, hello, i have seen your replies in all similar threads. what other paths can be taken? I know that there are clustered file systems. But I don't know what are the benefits. It may also be possible to refuse the general storage, only in favor of which option?​

the option in the choice of storage is obtained only when the machine is turned on, what is the feature here?

the virtual machine has migrated, but the disks are now in raw format. is this a feature of shared lvm? so you can't take snapshots?

 

[bbgeek17]

hello, i have seen your replies in all similar threads. what other paths can be taken?

you will need to define your requirements, infrastructure limitations, goals and use-case (lab vs commercial) to answer that question.

I know that there are clustered file systems.

compared to LVM (block), the CFS (file) can be more flexible and provide larger set of features and functions. However, its not integrated in any way in PVE.

the option in the choice of storage is obtained only when the machine is turned on, what is the feature here?

The "target storage" appears only when VM is started _and_ you have local storage (resource) it seems. I dont think its a "feature" per-say. Probably PVE developers felt that this would be most common use case without over-complicating UI . As I said you can migrate on a disk by disk basis, as well as use CLI.

the virtual machine has migrated, but the disks are now in raw format.

PVE can store VM disks in "raw" formatted file. Meaning that it has less structure and metadata than qcow, for example. Essentially this file format emulates raw disk storage, which is how any physical disk stores blocks. In the case of LVM (thin or thick) the disks are always in raw format. The snapshots in thin LVM come with LVM ability to allocate metadata on the fly as needed.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

P.S. edited to expand use case description

 

[rusquad]

you will need to define your requirements, infrastructure limitations, goals and use-case (lab vs commercial) to answer that question.

thanks for the detailed answer. I don’t know how commercial this is, but yes, services for the site will work there. I need to create a working cluster with two servers and fiber channel storage. I can add a 3rd server to the cluster as a member, but it can't access the storage (it's just far away). Accordingly, I need backups, snapshots, live migration on a one storage for 2 main servers (so far for 2).

 

[bbgeek17]

cluster+FC shared storage+snapshots=clustered filesystem.
Take a look at OCFS2 or similar products.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[rusquad]

cluster+FC shared storage+snapshots=clustered filesystem.
Take a look at OCFS2 or similar products.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

is a cluster file system added to the storage using a directory?

 

[bbgeek17]

is a cluster file system added to the storage using a directory?

yes, your VMs will use qcow on top of the filesystem.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[rusquad]

thanks, ocfs2 works great!

 

[rusquad]

when migrating between nodes, the transfer of RAM goes through the network. is it possible to set it up differently?

 

[fabian]

no. you can select which network/ip range to use, and whether to use SSH or not, but there are no other transport options.