Cluster, SAN and online migration, how can I give a direct-SAN-disk to a vm?

[Gianni]

Hi,
I have a two nodes cluster connected to a Fibre-Channell SAN to store vm's raw disk, I use LVM storage model.
All work fine. I have many vm and I can online migrate the vm from node1 to node2 and vice versa.

But now I need give a direct-SAN-disk to a windows 2003 vm.
No problem, I gave the command:

qm set 110 -ide1 /dev/mapper/SANDisk4

this command append to the 110.conf this line:

ide1: /dev/mapper/SANDisk4

The VM work fine and I can use the new NTFS SAN-disk.
But if I try to migrate the vm from node1 to node2 I get an error, the direct-SAN-disk is considered as local disk and is not shared.
This is the error:

/usr/sbin/qmigrate --online 192.168.10.49 110
Mar 03 16:29:54 starting migration of VM 110 to host '192.168.10.49'
Mar 03 16:29:54 copying disk images
Mar 03 16:29:54 Failed to sync data - cant migrate local file/device '/dev/mapper/SANDisk4
Mar 03 16:29:54 migration aborted
VM 110 migration failed -


How can I give a direct-SAN-disk to a vm and declare it shared to use the online migration?

Thanks in advance.

 

[dietmar]

How can I give a direct-SAN-disk to a vm and declare it shared to use the online migration?

That does not work. But why don't you use LVM?

Maybe it is possible to define an iSCSI storage, and then use that in the vm directly - but never tried that.

 

[Gianni]

That does not work. But why don't you use LVM?

It's more complicated, I must use a direct disk because I give the SAN-snapshots of this disk to a physical Windows server that don't read the LVM, but only NTFS.

Maybe it is possible to define an iSCSI storage, and then use that in the vm directly - but never tried that.

My SAN (Hp Eva4400) use Fibre Channel protocol and I can't define a iscsi storage model, I see my SAN-disk as internal logical disk.
Example: SANDisk4

# multipath -ll
SANDisk4 ( MyWWID-Disk ) dm-0 HP ,HSV200
[size=30G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=120][active]
\_ 3:0:0:3 sdi 8:128 [active][ready]
\_ 2:0:0:3 sdc 8:32 [active][ready]
\_ 2:0:1:3 sdf 8:80 [active][ready]
\_ 3:0:1:3 sdl 8:176 [active][ready]

# ls -al /dev/mapper/
[...]
brw-rw---- 1 root disk 251, 1 15 feb 10:38 SANDisk4
brw-rw---- 1 root disk 251, 2 15 feb 10:38 SANDisk2
brw-rw---- 1 root disk 251, 13 4 mar 08:00 SANDisk2-vm--107--disk--1
brw-rw---- 1 root disk 251, 11 24 feb 11:33 SANDisk2-vm--108--disk--1
brw-rw---- 1 root disk 251, 14 4 mar 08:01 SANDisk2-vm--110--disk--1

I have tried to define a "dir" model shared in storage.conf, but it use in the /etc/qemu-server/110.conf result syntax error and the vm don't start:

/etc/pve/storage.cfg

[...]
dir: DirectDisk
path /dev/mapper
content images


/etc/qemu-server/110.conf

name: vmWin2k3
ide2: none,media=cdrom
vlan0: virtio=7E:5E:51:46:0A:5D
bootdisk: ide0
ostype: w2k3
memory: 1024
onboot: 1
sockets: 1
cores: 1
ide0: SANDisk2:vm-110-disk-1
ide1: DirecDisk:SANDisk4

The DirecDisk:SANDisk4 is presented to both cluster's nodes and obviously shared.
It's used only from the vm 110.

How can I define it shared?
Where I can find documentation for vm config file, storage.conf?

 

[dietmar]

How can I define it shared?

I guess that is currently not possible. Don't know what would be the best way to allow that - any ideas?

 

[Gianni]

I guess that is currently not possible. Don't know what would be the best way to allow that - any ideas?

Yes, I would define it in storage.cfg as another storage model, like this:

direct: DirectDisk
path /dev/mapper/SANDisk4
shared
content none

This storage model cannot be used by proxmox, it is only passed to the VM as is.
When I migrate the VM, proxmox should process the disk in the same way as a shared LVM volume, letting the other cluster node use the disk when it takes control of the VM.

 

[dietmar]

Yes, that sounds good - i will add that to the todo list for 2.0