ceph volumes and --image-shared

[trekkygeek]

hello, I'm trying to work on a centos pacemaker cluster. I want to mount a gfs2 fS coming from a volume shared to two VM. I created the volume on proxmox1 for vm1, then modified vm2 conf file on proxmox2. Starting both VM goes without issues. both then see the same disk. The problem is that nothing can really be shared. I can create and mount the gfs2 FS from VM1 but VM2 complains that "wrong fs type", etc.

I'm suspecting something with the rbd backend.

Am I the only one that is trying to share a volume to multiple VM's?

 

[spirit]

Hi,
I'm doing it with ocfs2 without problem.

but If you use ceph jewel, since this release. they are a exclusive-lock enabled.

you can disable it your on ceph cluster with:

#rbd feature disable pool/vm-100-disk-1 exclusive-lock, object-map, fast-diff

 

[trekkygeek]

Thank you for your reply.
Am I doing it wrong?

root@r7101:~# ceph --version
ceph version 0.94.9 (fe6d859066244b97b24f09d46552afc2071e6f90)
root@r7101:~# rbd feature disable vm-107-disk-3 exclusive-lock, object-map, fast-diff
rbd: error parsing command 'feature'; -h or --help for usage

I guess they changed the arguments. I'm checking the man page.

 

[trekkygeek]

 

[trekkygeek]

Found something interresting. When both VM are on the same proxmox host, it works.

 

[trekkygeek]

back to drawing board....it just doesn't work.

[root@dev-pcmk-1 ~]# parted /dev/vdd -a optimal
GNU Parted 3.1
Using /dev/vdd
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mkpart primary 0 100%
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? i
(parted) q
Information: You may need to update /etc/fstab.

[root@dev-pcmk-1 ~]# blkid
/dev/vda1: UUID="5bd70845-7daf-4e7a-b9ac-d3720b358dc7" TYPE="xfs"
/dev/vda2: UUID="Zyr6Fy-vYVe-Ii4c-5Ef9-mGGe-G2Sh-E0aQ9B" TYPE="LVM2_member"
/dev/mapper/rootvg-root: UUID="542a6d25-7b8e-493a-927d-b67aadb6263a" TYPE="xfs"
/dev/mapper/rootvg-swap: UUID="a8521965-5cf4-40e7-a15e-9cd804c5908b" TYPE="swap"
/dev/vdc1: PARTLABEL="primary" PARTUUID="9346bc63-1f01-41ef-ac32-b6080262e475"
/dev/vdd1: PARTLABEL="primary" PARTUUID="30f66408-4a56-4018-8f4f-18946f2e4981"


[root@dev-pcmk-2 ~]# reboot
Using username "root".
Authenticating with public key "rsa-key-20150114" from agent
Last login: Thu Nov 10 09:10:18 2016 from 10.3.1.109
[root@dev-pcmk-2 ~]# blkid
/dev/vda1: UUID="5bd70845-7daf-4e7a-b9ac-d3720b358dc7" TYPE="xfs"
/dev/vda2: UUID="Zyr6Fy-vYVe-Ii4c-5Ef9-mGGe-G2Sh-E0aQ9B" TYPE="LVM2_member"
/dev/vdc: LABEL="dev-pcmk:test" UUID="c03a5cb9-db4a-993b-e640-3b2c5b912d0f" TYPE="gfs2"
/dev/mapper/rootvg-root: UUID="542a6d25-7b8e-493a-927d-b67aadb6263a" TYPE="xfs"
/dev/vdd: PTTYPE="gpt"
/dev/mapper/rootvg-swap: UUID="a8521965-5cf4-40e7-a15e-9cd804c5908b" TYPE="swap"

 

[trekkygeek]

Got it...stupid error on my side. I did a cut and paste in the qemu VM conf file to try to same 5 seconds...actually lost about two days..


the virtio1 was before base-105-disk-1/vm-107-disk-2 instead of just vm-107-disk-2...

 

[spirit]

Thank you for your reply.
Am I doing it wrong?

root@r7101:~# ceph --version
ceph version 0.94.9 (fe6d859066244b97b24f09d46552afc2071e6f90)
root@r7101:~# rbd feature disable vm-107-disk-3 exclusive-lock, object-map, fast-diff
rbd: error parsing command 'feature'; -h or --help for usage

I guess they changed the arguments. I'm checking the man page.

for hammer ,this is ok, you don't need this command