Moving VM's storage to iSCSI: "can't allocate space in iscsi storage"

[Zbig]

Hello

I want each separate LUN under my iSCSI target to serve as a raw storage for a single VM disk image. I have added iscsi storage making sure to check the "Use LUNs directly" option.

What I can do:

• Create a VM with iSCSI LUN (whether empty or not) attached as its virtual disk
• Install an OS on the LUN-backed disk and run the VM from it
• Move the disk from iSCSI to another storage like local ZFS

What I cannot do:

• Move a disk image from another storage to iSCSI one. Every time I try this I'm getting an error "storage migration failed: can't allocate space in iscsi storage"

I understand that iSCSI doesn't have any notion of data structure or file system but that's why I chose to manually create a new LUN for each VM virtual disk on Synology DSM level. I don't want to have one big LUN and slap an LVM on top of it.

Is there any solution for that?

Best regards,
Zbig

 

[bbgeek17]

you can use "qm disk move " (man qm) which allows you to specify target disk, which would be pre-created by you

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Zbig]

you can use "qm disk move " (man qm) which allows you to specify target disk, which would be pre-created by you

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Thank you for your response. How do I specify LUN as part of the [<storage>] parameter of the qm disk move <vmid> <disk> [<storage>] [OPTIONS] command? Manpage doesn't elaborate on that and each guess of the many I took resulted in error.

 

[bbgeek17]

what does your VM config look like now? "qm config"

Does this information from "man" not help at all?

 --target-disk <efidisk0 | ide0 | ide1 | ide2 | ide3 | sata0 | sata1 | sata2 | sata3 | sata4 | sata5 | scsi0 | scsi1 | scsi10 | scsi11 | scsi12 | scsi13 | scsi14 | scsi15 | scsi16 | scsi17 |
       scsi18 | scsi19 | scsi2 | scsi20 | scsi21 | scsi22 | scsi23 | scsi24 | scsi25 | scsi26 | scsi27 | scsi28 | scsi29 | scsi3 | scsi30 | scsi4 | scsi5 | scsi6 | scsi7 | scsi8 | scsi9 | tpmstate0 |
       unused0 | unused1 | unused10 | unused100 | unused101 | unused102 | unused103 | unused104 | unused105 | unused106 | unused107 | unused108 | unused109 | unused11 | unused110 | unused111 |
       unused112 | unused113 | unused114 | unused115 | unused116 | unused117 | unused118 | unused119 | unused12 | unused120 | unused121 | unused122 | unused123 | unused124 | unused125 | unused126 |
       unused127 | unused128 | unused129 | unused13 | unused130 | unused131 | unused132 | unused133 | unused134 | unused135 | unused136 | unused137 | unused138 | unused139 | unused14 | unused140 |
       unused141 | unused142 | unused143 | unused144 | unused145 | unused146 | unused147 | unused148 | unused149 | unused15 | unused150 | unused151 | unused152 | unused153 | unused154 | unused155 |
       unused156 | unused157 | unused158 | unused159 | unused16 | unused160 | unused161 | unused162 | unused163 | unused164 | unused165 | unused166 | unused167 | unused168 | unused169 | unused17 |
       unused170 | unused171 | unused172 | unused173 | unused174 | unused175 | unused176 | unused177 | unused178 | unused179 | unused18 | unused180 | unused181 | unused182 | unused183 | unused184 |
       unused185 | unused186 | unused187 | unused188 | unused189 | unused19 | unused190 | unused191 | unused192 | unused193 | unused194 | unused195 | unused196 | unused197 | unused198 | unused199 |
       unused2 | unused20 | unused200 | unused201 | unused202 | unused203 | unused204 | unused205 | unused206 | unused207 | unused208 | unused209 | unused21 | unused210 | unused211 | unused212 |
       unused213 | unused214 | unused215 | unused216 | unused217 | unused218 | unused219 | unused22 | unused220 | unused221 | unused222 | unused223 | unused224 | unused225 | unused226 | unused227 |
       unused228 | unused229 | unused23 | unused230 | unused231 | unused232 | unused233 | unused234 | unused235 | unused236 | unused237 | unused238 | unused239 | unused24 | unused240 | unused241 |
       unused242 | unused243 | unused244 | unused245 | unused246 | unused247 | unused248 | unused249 | unused25 | unused250 | unused251 | unused252 | unused253 | unused254 | unused255 | unused26 |
       unused27 | unused28 | unused29 | unused3 | unused30 | unused31 | unused32 | unused33 | unused34 | unused35 | unused36 | unused37 | unused38 | unused39 | unused4 | unused40 | unused41 | unused42
       | unused43 | unused44 | unused45 | unused46 | unused47 | unused48 | unused49 | unused5 | unused50 | unused51 | unused52 | unused53 | unused54 | unused55 | unused56 | unused57 | unused58 |
       unused59 | unused6 | unused60 | unused61 | unused62 | unused63 | unused64 | unused65 | unused66 | unused67 | unused68 | unused69 | unused7 | unused70 | unused71 | unused72 | unused73 | unused74
       | unused75 | unused76 | unused77 | unused78 | unused79 | unused8 | unused80 | unused81 | unused82 | unused83 | unused84 | unused85 | unused86 | unused87 | unused88 | unused89 | unused9 |
       unused90 | unused91 | unused92 | unused93 | unused94 | unused95 | unused96 | unused97 | unused98 | unused99 | virtio0 | virtio1 | virtio10 | virtio11 | virtio12 | virtio13 | virtio14 | virtio15
       | virtio2 | virtio3 | virtio4 | virtio5 | virtio6 | virtio7 | virtio8 | virtio9>
           The config key the disk will be moved to on the target VM (for example, ide0 or scsi1). Default is the source disk key.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[bbgeek17]

You also can move the disk to qcow or raw type file, then use qemu-img or dd to transfer it to your LUN directly.

good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Zbig]

My VM config looks like this:

root@pve-1:~# qm config 150
agent: 1
boot: order=scsi0
cores: 1
memory: 680
name: PostgreSQL
net0: vmxnet3=00:50:56:8d:e7:f7,bridge=vmbr1,firewall=1,tag=20
scsi0: local-zfs:vm-150-disk-0,size=10G
scsihw: pvscsi
smbios1: uuid=c3b8633c-14a7-4597-a5ce-59766d1dca81
vmgenid: 18e12ff9-d786-4307-8042-c8204ddc91e3

regarding --target-disk: the way I understand it, this parameter is used to specify which virtual interface of the target VM you'd like to attach the disk being moved to in case you're moving disks between VMs - not relevant in my case.

Here's like my nas1-iscsi storage looks like currently:

root@pve-1:~# pvesm list nas1-iscsi
Volid                                                   Format  Type             Size VMID
nas1-iscsi:0.0.1.scsi-360014057e7e5bedd8a02d4fd9db2b0d1 raw     images    26843545600

I have tried various versions of this:

qm disk move 150 scsi0 nas1-iscsi

but the "nas1-iscsi" alone isn't precise enough as you have to include LUN information somehow but my every attempt to do so resulted in error about illegal characters.

 

[bbgeek17]

I've taken a closer look at this and did a few experiments. Looks like there is no way to migrate data from any disk to a Direct iSCSI Lun via PVE. You are going to have to workaround it by doing a direct copy. I.e. attach LUN to host, then either use qemu-img, or dd or similar tool.

Good luck.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Zbig]

Thank you very much for taking time to test it on your side. Much appreciated. I'll explore the workarounds you've suggested.

Have a good one