That's an overview of my architecture. Toe is a server I use to achieve quorum but I'm not sure if I even need it for that, but that's not the point of this post. I'm concerned about the shared storage volume (DELL MD3420). As you'll see below, I'm not sure if I have my LVM configuration set up correctly. Tic and Tac can access the same volumes, but they don't update in real time.
Here is my configuration from the web interface:
shared-storage1 is sda/sdc in the above and proxmox-data is sdb/sdd.
Code:
root@tac:/# pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/3600a098000e05de6000004b95d97ed71 shared-storage1 lvm2 a-- 5.43t 4.63t
/dev/mapper/3600a098000e05de6000004bc5d97ed72 shared-storage2 lvm2 a-- 5.43t <131.57g
/dev/sde3 pve lvm2 a-- 223.00g <16.00g
root@tac:/# vgs
VG #PV #LV #SN Attr VSize VFree
pve 1 3 0 wz--n- 223.00g <16.00g
shared-storage1 1 16 0 wz--n- 5.43t 4.63t
shared-storage2 1 1 0 wz--n- 5.43t <131.57g
root@tac:/# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data pve twi-aotz-- 140.39g 0.00 1.14
root pve -wi-ao---- 55.75g
swap pve -wi-ao---- 8.00g
vm-100-disk-0 shared-storage1 -wi-a----- 40.00g
. . .
vm-114-disk-0 shared-storage1 -wi-a----- 40.00g
vm-115-disk-0 shared-storage1 -wi-ao---- 8.00g
proxmox-data shared-storage2 -wi-ao---- 5.30tThe output of those LVM commands are the same on my other server, tic, except for the "lvs" command.
Bash:
root@tic:/etc/lvm# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data pve twi-a-tz-- 140.39g 0.00 1.14
root pve -wi-ao---- 55.75g
swap pve -wi-ao---- 8.00g
vm-100-disk-0 shared-storage1 -wi------- 40.00g
. . .
vm-114-disk-0 shared-storage1 -wi-ao---- 40.00g
vm-115-disk-0 shared-storage1 -wi------- 8.00g
proxmox-data shared-storage2 -wi-ao---- 5.30tTic doesn't have the a in the 5th attribute spot.
Some other relevant commands:
Bash:
root@tac:/# multipath -ll
3600a098000e05de6000004bc5d97ed72 dm-7 DELL,MD34xx
size=5.4T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 0:0:1:1 sdd 8:48 active ready running
`-+- policy='service-time 0' prio=9 status=enabled
`- 0:0:0:1 sdb 8:16 active ready running
3600a098000e05de6000004b95d97ed71 dm-6 DELL,MD34xx
size=5.4T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 0:0:0:0 sda 8:0 active ready running
`-+- policy='service-time 0' prio=9 status=enabled
`- 0:0:1:0 sdc 8:32 active ready running
root@tic:/etc/lvm# multipath -ll
3600a098000e05de6000004bc5d97ed72 dm-6 DELL,MD34xx
size=5.4T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 0:0:3:1 sdb 8:16 active ready running
`-+- policy='service-time 0' prio=9 status=enabled
`- 0:0:2:1 sdd 8:48 active ready running
3600a098000e05de6000004b95d97ed71 dm-5 DELL,MD34xx
size=5.4T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 0:0:2:0 sdc 8:32 active ready running
`-+- policy='service-time 0' prio=9 status=enabled
`- 0:0:3:0 sda 8:0 active ready running
Bash:
root@tac:/# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 5.4T 0 disk
└─3600a098000e05de6000004b95d97ed71 253:6 0 5.4T 0 mpath
├─shared--storage1-vm--106--disk--0 253:8 0 20G 0 lvm
├─shared--storage1-vm--107--disk--0 253:9 0 100G 0 lvm
├─shared--storage1-vm--108--disk--0 253:10 0 50G 0 lvm
├─shared--storage1-vm--100--disk--0 253:11 0 40G 0 lvm
├─shared--storage1-vm--100--disk--1 253:12 0 40G 0 lvm
├─shared--storage1-vm--101--disk--0 253:13 0 40G 0 lvm
├─shared--storage1-vm--102--disk--0 253:14 0 50G 0 lvm
├─shared--storage1-vm--103--disk--0 253:15 0 50G 0 lvm
├─shared--storage1-vm--109--disk--0 253:16 0 20G 0 lvm
├─shared--storage1-vm--110--disk--0 253:17 0 50G 0 lvm
├─shared--storage1-vm--111--disk--0 253:18 0 100G 0 lvm
├─shared--storage1-vm--112--disk--0 253:19 0 100G 0 lvm
├─shared--storage1-vm--113--disk--0 253:20 0 8G 0 lvm
├─shared--storage1-vm--114--disk--0 253:21 0 40G 0 lvm
├─shared--storage1-vm--104--disk--0 253:22 0 100G 0 lvm
└─shared--storage1-vm--115--disk--0 253:23 0 8G 0 lvm
sdb 8:16 0 5.4T 0 disk
└─3600a098000e05de6000004bc5d97ed72 253:7 0 5.4T 0 mpath
└─shared--storage2-proxmox--data 253:24 0 5.3T 0 lvm /proxmox-data
sdc 8:32 0 5.4T 0 disk
└─3600a098000e05de6000004b95d97ed71 253:6 0 5.4T 0 mpath
├─shared--storage1-vm--106--disk--0 253:8 0 20G 0 lvm
├─shared--storage1-vm--107--disk--0 253:9 0 100G 0 lvm
├─shared--storage1-vm--108--disk--0 253:10 0 50G 0 lvm
├─shared--storage1-vm--100--disk--0 253:11 0 40G 0 lvm
├─shared--storage1-vm--100--disk--1 253:12 0 40G 0 lvm
├─shared--storage1-vm--101--disk--0 253:13 0 40G 0 lvm
├─shared--storage1-vm--102--disk--0 253:14 0 50G 0 lvm
├─shared--storage1-vm--103--disk--0 253:15 0 50G 0 lvm
├─shared--storage1-vm--109--disk--0 253:16 0 20G 0 lvm
├─shared--storage1-vm--110--disk--0 253:17 0 50G 0 lvm
├─shared--storage1-vm--111--disk--0 253:18 0 100G 0 lvm
├─shared--storage1-vm--112--disk--0 253:19 0 100G 0 lvm
├─shared--storage1-vm--113--disk--0 253:20 0 8G 0 lvm
├─shared--storage1-vm--114--disk--0 253:21 0 40G 0 lvm
├─shared--storage1-vm--104--disk--0 253:22 0 100G 0 lvm
└─shared--storage1-vm--115--disk--0 253:23 0 8G 0 lvm
sdd 8:48 0 5.4T 0 disk
└─3600a098000e05de6000004bc5d97ed72 253:7 0 5.4T 0 mpath
└─shared--storage2-proxmox--data 253:24 0 5.3T 0 lvm /proxmox-data
sde 8:64 0 223.5G 0 disk
. . .
root@tic:/etc/lvm# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 5.4T 0 disk
└─3600a098000e05de6000004b95d97ed71 253:5 0 5.4T 0 mpath
├─shared--storage1-vm--100--disk--0 253:9 0 30G 0 lvm
├─shared--storage1-vm--107--disk--0 253:10 0 100G 0 lvm
├─shared--storage1-vm--112--disk--0 253:14 0 100G 0 lvm
├─shared--storage1-vm--102--disk--0 253:16 0 50G 0 lvm
├─shared--storage1-vm--109--disk--0 253:18 0 20G 0 lvm
├─shared--storage1-vm--111--disk--0 253:19 0 100G 0 lvm
└─shared--storage1-vm--114--disk--0 253:22 0 40G 0 lvm
sdb 8:16 0 5.4T 0 disk
└─3600a098000e05de6000004bc5d97ed72 253:6 0 5.4T 0 mpath
└─shared--storage2-proxmox--data 253:13 0 5.3T 0 lvm /proxmox-data
sdc 8:32 0 5.4T 0 disk
└─3600a098000e05de6000004b95d97ed71 253:5 0 5.4T 0 mpath
├─shared--storage1-vm--100--disk--0 253:9 0 30G 0 lvm
├─shared--storage1-vm--107--disk--0 253:10 0 100G 0 lvm
├─shared--storage1-vm--112--disk--0 253:14 0 100G 0 lvm
├─shared--storage1-vm--102--disk--0 253:16 0 50G 0 lvm
├─shared--storage1-vm--109--disk--0 253:18 0 20G 0 lvm
├─shared--storage1-vm--111--disk--0 253:19 0 100G 0 lvm
└─shared--storage1-vm--114--disk--0 253:22 0 40G 0 lvm
sdd 8:48 0 5.4T 0 disk
└─3600a098000e05de6000004bc5d97ed72 253:6 0 5.4T 0 mpath
└─shared--storage2-proxmox--data 253:13 0 5.3T 0 lvm /proxmox-data
sde 8:64 0 223.5G 0 disk
. . .I know that's a lot of information, but I'm pretty new to this and just want to make sure I have it set up correctly. As of right now VMs work fine and can be migrated between Tic and Tac, but the "proxmox-data" LVM isn't updating on both nodes even though they are both connected to it. The only way it works is if I reboot the host and then everything is updated.
Thanks in advance.
