Help with moving storage: From LVM to iSCSI

[NateNate]

TL;DR: Getting error trying to move Local-LVM to iSCSI.
Getting error

create full clone of drive scsi0 (local-lvm:vm-102-disk-1)
TASK ERROR: storage migration failed: can't allocate space in iscsi storage

Short Story Long:
New to Proxmox, but have been expanding my knowledge of it thanks to vmware's expensive ecosystem.
In my test environment, I have a fairly simple setup: 2 Nodes, connected via iSCSI to TrueNAS Server (10G connection)

No issues creating traditional VMs, but having all kinds of fits with HomeAssistant OS.
Tried numerous of times to get it to install on the iSCSI Share with no success..

Finally installed it locally on one of the hosts, and trying to move storage from the Local Node to my NAS where it have the inherited resilience

 

[bbgeek17]

It seems like you are trying to move data to "raw" iSCSI storage. You can't do this - there needs to be an LVM layer on top of the iSCSI.

You may want to read through this article: https://kb.blockbridge.com/technote/proxmox-lvm-shared-storage/

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[NateNate]

It seems like you are trying to move data to "raw" iSCSI storage. You can't do this - there needs to be an LVM layer on top of the iSCSI.

You may want to read through this article: https://kb.blockbridge.com/technote/proxmox-lvm-shared-storage/

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

You are correct, this is Raw storage. I will be following the link you sent and tinker with this. Once setup, all Nodes access this for HA?

 

[bbgeek17]

Once setup, all Nodes access this for HA?

I can't be certain until you share you final configuration. In theory - yes.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[NateNate]

Thanks for the pointer! Got it working, Partially...
Like you said, from the datacenter I created an LVM on top of the iSCSI RAW.
Then from the console of Node1, i ran the HomeAssistant Script and pointed it to this LVM, and it worked great.

Issue is, only one of the nodes can "See" this new LVM. It creates it on both, but node2 has a question mark on it.

If i try to Migrate the newly created VM from Node 1 to Node 2, it fails.

I was connected to the IP of Node one but ran the commands in the UI under the Datacenter portion.
I started over and did this from the IP of Node2 just for kicks, but getting this error when I try to create the LVM, again from Datacenter but while on the IP of node 2:

create storage failed: command '/sbin/pvs --separator : --noheadings --units k --unbuffered --nosuffix --options pv_name,pv_size,vg_name,pv_uuid /dev/disk/by-id/scsi-36589cfc00000082597ae718f0d25b58a' failed: exit code 5 (500)

I have tried a combination of Use LUNs Directly (yes/no) and shared LVM Shared (yes/no).

Thanks!

 

[bbgeek17]

First, do the following:
ensure that your iSCSI storage pool is present on all nodes.
ensure that you can list the LUNs from all nodes
if you need multipath - configure it
LVM will not work if multiple paths are present but multipath is not configured.

https://kb.blockbridge.com/technote/proxmox-lvm-shared-storage/

If it continues to not work for you, provide the output of:
cat /etc/pve/storage.cfg
pvesm status (both nodes)
pvesm iscsiscan (both nodes)
pvesm list [iscsi pool] (both nodes)
pvs , vgs , lvs (both nodes)

used CODE tags from the edit box menu

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[NateNate]

Well, i rebooted Node2 and it worked. lol
(have you tried turning it off and on again)