how to make lvm-thin local storage shared?

[Mr.Gosh]

we have three nodes in a cluster
all are using nfs shared storage but for io intense vms we use local ssd mirror raids as a faster storage.
but what to do if i want to migrate a vm from one local ssd lvm-thin storage to another node without having to move the storage to the nfs before this?

is there a possibility to make a local lvm-thin shared after creation?

after some more reading - could a local iscsi target at every pve cluster node be a possible solution?

 

[dcsapak]

first: the shared option is meant to indicate that the same storage is available to the selected nodes, meaning exactly the same (think nfs server, iscsi server etc.)

if you want to live migrate from one local storage to another node: this is (currently) not possible (except the method you already mentioned: moving the disk to shared storage, live migrate, move disk back to local)

if you want to migrate them offline: the patches for lvm-thin should not be far away AFAIK

 

[Mr.Gosh]

Thanks for clarifying this.
But why does the creation of a local lvm-thin on one device lead to a creation of these local lvm-thin devices on the other nodes? this is disturbing...
I can only identify on which machine this is really available through the summary page:
Here the local lvm-thin that is available on node pve00

and here the lvm-thin from pve02 "summaried by pve00:

 

[dcsapak]

when adding a storage via Datacenter -> storage you can specify the nodes on which the storage is available
see

there you can restrict the storages to the nodes where they belong

 

[Mr.Gosh]

Thanks that makes things clearer - and what about the "local iscsi" stuff? is this something that could work to move things directly between local storages?

 

[dcsapak]

what do you mean with "local iscsi"? iscsi is a protocol to send scsi commands over the network, so to share a block device.
again, if you want online migration between nodes, you need a shared storage to which every node has access

 

[Mr.Gosh]

what i meant was something like this:

At the moment it looks like this:

• pve00 has a local 1tb sdd raid1 and some not io intense stuff via nfs (small vms, backup etc.)

• pve01 has a raid6 out of 6x5tb HDDs and the same nfs acces like above

if i want to migrate a vm from the raid6 to the raid1 i have to move the vm-disk to the nfs - than migrate the vm to pve00 than move the hdd again to the raid1

My Questioan was if something like this could be possible:

• pve00 has a local 1tb sdd raid1 and some not io intense stuff via nfs (small vms, backup etc.)
  • here could be a iscsi target running on top of the 1tb ssd raid1

• pve01 has a raid6 out of 6x5tb HDDs and the same nfs acces like above
  • here could be a iscsi target running on top of the raid6

in this setup all the clients could be able to live-migrate things through their semi-local iscsi targets and it would be as easy to setup as a ceph cluster without the necessary hardware overhead

perhaps i am completely wrong here - but this looks like realistic setup for me... ;-) - what do you think?

 

[LnxBil]

Yes, you can export all your local disks via iSCSI to all other nodes (including the one that is exporting the data) to have a "real" shared storage. There is nothing directly available in Proxmox to do this, yet you can install the target software and install it the Debian way.

Keep in mind: This is no HA setup and if you loose one node, all VMs on this particular storage are loosing their storage and will most probably crash.

 

[alamadrid]

first: the shared option is meant to indicate that the same storage is available to the selected nodes, meaning exactly the same (think nfs server, iscsi server etc.)

if you want to live migrate from one local storage to another node: this is (currently) not possible (except the method you already mentioned: moving the disk to shared storage, live migrate, move disk back to local)

if you want to migrate them offline: the patches for lvm-thin should not be far away AFAIK

This post is from 2016, has Proxmox released a way to offline migrate VMs between local LVM-Thin storage on different servers?

 

[dcsapak]

This post is from 2016, has Proxmox released a way to offline migrate VMs between local LVM-Thin storage on different servers?

offline migration works since a long time, the thread is about live migration

 

[vanttech]

Cherry on top and 2 years later, Proxmox 7 still nothing to make live migration happen?

 

[fabian]

Cherry on top and 2 years later, Proxmox 7 still nothing to make live migration happen?

live migration with local disks works for quite a while already.

 

[vanttech]

It works yes but performance is hurtful to say the least

 

[fabian]

well... for live migration with local disks you need to copy all of the disks' content over the network while simultaneously reading/writing it, so yes, this is quite the intensive operation..