pvcreate /dev/drbd0 - device not found, software raid, drbd and lvm

whitewater

Member
Nov 26, 2012
107
0
16
france
Hello,
i have two proxmox node, version 3.2.
i have install proxmox on a fresh debian install, to have software raid.

On each node, i would like to use :
- sda & sdb for OS raid volume, in software raid. md0 & md1 volume.
- sdc & sdd for VM raid volume, in software raid. md2 volume.
- DRBD for sync VM raid volume between the two node, with a dedicated link.

At this moment, 2 VM run well on node 1.

My problem is, i can't create drbd volume :
Code:
root@proxmox02:~# pvcreate /dev/drbdr0
  Device /dev/drbdr0 not found (or ignored by filtering).


drbd is ok :
Code:
root@proxmox02:~# cat /proc/drbd
version: 8.3.13 (api:88/proto:86-96)
GIT-hash: 83ca112086600faacab2f157bc5a9324f7bd7f77 build by root@sighted, 2012-10-09 12:47:51
 0: cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate C r-----
    ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:d oos:0


drbd ressource file :
Code:
resource r0 {
  protocol C;
  startup {
    wfc-timeout  15;     # non-zero wfc-timeout can be dangerous (http://forum.proxmox.com/threads/3465-Is-it-safe-to-use-wfc-timeout-in-DRBD-configuration)
    degr-wfc-timeout 60;
    become-primary-on both;
  }
  net {
    cram-hmac-alg sha1;
    shared-secret "my-secret";
    allow-two-primaries;
    after-sb-0pri discard-zero-changes;
    after-sb-1pri discard-secondary;
    after-sb-2pri disconnect;
  }
  on proxmox02 {            
    device /dev/drbd0;        
    disk /dev/md2;            
    address 192.168.192.1:7788;    
    meta-disk internal;
  }
  on proxmox03 {            
    device /dev/drbd0;        
    disk /dev/md2;            
    address 192.168.192.2:7788;    
    meta-disk internal;
  }
}



lvm.conf filter :
i have tested with :
Code:
filter = [ "a/.*/" ]


or


Code:
filter = [ "a|drbd.*|", "r|.*|" ]


no success.


here some information :
Code:
root@proxmox02:~# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdc1[0] sdd1[1]
      1943227200 blocks super 1.2 [2/2] [UU]




md1 : active raid1 sda2[0] sdb2[1]
      1855337280 blocks super 1.2 [2/2] [UU]




md0 : active raid1 sda1[0] sdb1[1]
      487104 blocks super 1.2 [2/2] [UU]


unused devices: <none>


Code:
root@proxmox02:~# df -h


Sys. fich.                       Taille Util. Dispo Uti% Monté sur
udev                                10M     0   10M   0% /dev
tmpfs                              1,6G  412K  1,6G   1% /run
/dev/mapper/pve-root                74G  1,4G   69G   2% /
tmpfs                              5,0M  4,0K  5,0M   1% /run/lock
tmpfs                              3,2G   47M  3,1G   2% /run/shm
/dev/md0                           461M   58M  380M  14% /boot
/dev/mapper/pve-data               1,6T   66G  1,5T   5% /var/lib/vz
/dev/fuse                           30M   20K   30M   1% /etc/pve
192.168.20.10:/volume3/vmproxmox   913G  310G  603G  34% /mnt/pve/synology
192.168.20.10:/volume1/proxmox     2,7T  1,5T  1,3T  54% /mnt/pve/synology-backup


have you got an idea ? thank you :D
 
Seems like your filter is wrong and order does matter.

You want to ignore any physical or logical devices that make up the DRBD volume.
Also, ignore any things that might be linked to those devices like /dev/disk

I *think* this is the filter statement you need:
Code:
filter = [ "r|/dev/md2|", "r|/dev/sdc1|", "r|/dev/sdd1|", "r|/dev/disk/|", "r|/dev/block/|", "a/.*/" ]
 
Here is :
Code:
root@proxmox02:~# ls -l /dev/drbd0
brw-rw---T 1 root disk 147, 0 juin   7 12:58 /dev/drbd0

Code:
root@proxmox02:~# ls -l /dev/drbd/by-res/
total 0
lrwxrwxrwx 1 root root 11 juin   7 12:58 r0 -> ../../drbd0

Thank you.
 
Last edited:
Here is the result :
Code:
root@proxmox02:~# pvcreate -vvv /dev/drbdr0 
Setting activation/monitoring to 0
        Processing: pvcreate -vvv /dev/drbdr0
        O_DIRECT will be used
      Setting global/locking_type to 1
      Setting global/wait_for_locks to 1
      File-based locking selected.
      Setting global/locking_dir to /run/lock/lvm
      Setting global/prioritise_write_locks to 1
      metadata/pvmetadataignore not found in config: defaulting to n
      metadata/pvmetadatasize not found in config: defaulting to 255
      metadata/pvmetadatacopies not found in config: defaulting to 1
      Locking /run/lock/lvm/P_orphans WB
        _do_flock /run/lock/lvm/P_orphans:aux WB
        _do_flock /run/lock/lvm/P_orphans WB
        _undo_flock /run/lock/lvm/P_orphans:aux
      /dev/drbdr0: stat failed: Aucun fichier ou dossier de ce type
        /dev/sda: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E81QCF: Aliased to /dev/sda in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E81QCF: Aliased to /dev/sda in device cache
        /dev/disk/by-id/wwn-0x5000c50066f2ff11: Aliased to /dev/sda in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0: Aliased to /dev/sda in device cache
        /dev/sda1: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E81QCF-part1: Aliased to /dev/sda1 in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E81QCF-part1: Aliased to /dev/sda1 in device cache
        /dev/disk/by-id/wwn-0x5000c50066f2ff11-part1: Aliased to /dev/sda1 in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0-part1: Aliased to /dev/sda1 in device cache
        /dev/sda2: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E81QCF-part2: Aliased to /dev/sda2 in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E81QCF-part2: Aliased to /dev/sda2 in device cache
        /dev/disk/by-id/wwn-0x5000c50066f2ff11-part2: Aliased to /dev/sda2 in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0-part2: Aliased to /dev/sda2 in device cache
        /dev/sdb: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E51NTG: Aliased to /dev/sdb in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E51NTG: Aliased to /dev/sdb in device cache
        /dev/disk/by-id/wwn-0x5000c500644c1e39: Aliased to /dev/sdb in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-1:0:0:0: Aliased to /dev/sdb in device cache
        /dev/sdb1: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E51NTG-part1: Aliased to /dev/sdb1 in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E51NTG-part1: Aliased to /dev/sdb1 in device cache
        /dev/disk/by-id/wwn-0x5000c500644c1e39-part1: Aliased to /dev/sdb1 in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-1:0:0:0-part1: Aliased to /dev/sdb1 in device cache
        /dev/sdb2: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E51NTG-part2: Aliased to /dev/sdb2 in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E51NTG-part2: Aliased to /dev/sdb2 in device cache
        /dev/disk/by-id/wwn-0x5000c500644c1e39-part2: Aliased to /dev/sdb2 in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-1:0:0:0-part2: Aliased to /dev/sdb2 in device cache
        /dev/sdc: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E81P8Y: Aliased to /dev/sdc in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E81P8Y: Aliased to /dev/sdc in device cache
        /dev/disk/by-id/wwn-0x5000c50066f32c19: Aliased to /dev/sdc in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-2:0:0:0: Aliased to /dev/sdc in device cache
        /dev/sdc1: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E81P8Y-part1: Aliased to /dev/sdc1 in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E81P8Y-part1: Aliased to /dev/sdc1 in device cache
        /dev/disk/by-id/wwn-0x5000c50066f32c19-part1: Aliased to /dev/sdc1 in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-2:0:0:0-part1: Aliased to /dev/sdc1 in device cache
        /dev/sdd: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E51QN6: Aliased to /dev/sdd in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E51QN6: Aliased to /dev/sdd in device cache
        /dev/disk/by-id/wwn-0x5000c500644bc31a: Aliased to /dev/sdd in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-3:0:0:0: Aliased to /dev/sdd in device cache
        /dev/sdd1: Added to device cache
        /dev/disk/by-id/ata-ST2000NC000-1CX164_Z1E51QN6-part1: Aliased to /dev/sdd1 in device cache
        /dev/disk/by-id/scsi-SATA_ST2000NC000-1CX_Z1E51QN6-part1: Aliased to /dev/sdd1 in device cache
        /dev/disk/by-id/wwn-0x5000c500644bc31a-part1: Aliased to /dev/sdd1 in device cache
        /dev/disk/by-path/pci-0000:00:1f.2-scsi-3:0:0:0-part1: Aliased to /dev/sdd1 in device cache
        /dev/drbd0: Added to device cache
        /dev/drbd/by-disk/md2: Aliased to /dev/drbd0 in device cache
        /dev/drbd/by-res/r0: Aliased to /dev/drbd0 in device cache
        /dev/loop0: Added to device cache
        /dev/loop1: Added to device cache
        /dev/loop2: Added to device cache
        /dev/loop3: Added to device cache
        /dev/loop4: Added to device cache
        /dev/loop5: Added to device cache
        /dev/loop6: Added to device cache
        /dev/loop7: Added to device cache
        /dev/ram0: Added to device cache
        /dev/ram1: Added to device cache
        /dev/ram10: Added to device cache
        /dev/ram11: Added to device cache
        /dev/ram12: Added to device cache
        /dev/ram13: Added to device cache
        /dev/ram14: Added to device cache
        /dev/ram15: Added to device cache
        /dev/ram2: Added to device cache
        /dev/ram3: Added to device cache
        /dev/ram4: Added to device cache
        /dev/ram5: Added to device cache
        /dev/ram6: Added to device cache
        /dev/ram7: Added to device cache
        /dev/ram8: Added to device cache
        /dev/ram9: Added to device cache
        /dev/dm-0: Added to device cache
        /dev/root: Aliased to /dev/dm-0 in device cache (preferred name)
        /dev/dm-1: Added to device cache
        /dev/dm-2: Added to device cache
        /dev/disk/by-id/dm-name-pve-data: Aliased to /dev/dm-2 in device cache (preferred name)
        /dev/disk/by-id/dm-uuid-LVM-PgKeQCkgDX4vB1m2aEsAXNgVsF0tL7c7MMrIj5RpknOom0Z2ai8mUOtuTzcuc83D: Aliased to /dev/disk/by-id/dm-name-pve-data in device cache
        /dev/disk/by-uuid/c8738227-f3b2-4e5f-b6dd-9011ffe4aa65: Aliased to /dev/disk/by-id/dm-name-pve-data in device cache
        /dev/mapper/pve-data: Aliased to /dev/disk/by-id/dm-name-pve-data in device cache (preferred name)
        /dev/pve/data: Aliased to /dev/mapper/pve-data in device cache (preferred name)
        /dev/md0: Added to device cache
        /dev/disk/by-id/md-name-proxmox02:0: Aliased to /dev/md0 in device cache
        /dev/disk/by-id/md-uuid-4bf9d448:d6c8d479:8142a9f3:1b1ee43d: Aliased to /dev/md0 in device cache
        /dev/disk/by-uuid/2e077ef4-e485-4915-b5fa-31d82948e676: Aliased to /dev/md0 in device cache
        /dev/md/0: Aliased to /dev/md0 in device cache
        /dev/md1: Added to device cache
        /dev/disk/by-id/md-name-proxmox02:1: Aliased to /dev/md1 in device cache
        /dev/disk/by-id/md-uuid-6ac8121c:345d385d:9a5a8216:2966da05: Aliased to /dev/md1 in device cache
        /dev/md/1: Aliased to /dev/md1 in device cache
        /dev/md2: Added to device cache
        /dev/disk/by-id/md-name-proxmox02:2: Aliased to /dev/md2 in device cache
        /dev/disk/by-id/md-uuid-dac374e7:769de5ae:014d9173:55080e79: Aliased to /dev/md2 in device cache
        /dev/md/2: Aliased to /dev/md2 in device cache
        /dev/md2p1: Added to device cache
        /dev/disk/by-id/md-name-proxmox02:2-part1: Aliased to /dev/md2p1 in device cache
        /dev/disk/by-id/md-uuid-dac374e7:769de5ae:014d9173:55080e79-part1: Aliased to /dev/md2p1 in device cache
        /dev/md/2p1: Aliased to /dev/md2p1 in device cache
      /dev/drbdr0: stat failed: Aucun fichier ou dossier de ce type
      Setting devices/sysfs_scan to 1
      Setting devices/md_component_detection to 1
      Setting devices/multipath_component_detection to 1
      Setting devices/ignore_suspended_devices to 0
      Setting devices/cache_dir to /run/lvm
      Setting devices/write_cache_state to 0
      /dev/drbdr0: stat failed: Aucun fichier ou dossier de ce type
  Device /dev/drbdr0 not found (or ignored by filtering).
      Unlocking /run/lock/lvm/P_orphans
        _undo_flock /run/lock/lvm/P_orphans
 
oupppsss... sorry ! :D
Here pvcreate -vvv /dev/drbd0 (no physical volume created)


Code:
root@proxmox02:~# pvcreate -vvv /dev/drbd0
Setting activation/monitoring to 0
        Processing: pvcreate -vvv /dev/drbd0
        O_DIRECT will be used
      Setting global/locking_type to 1
      Setting global/wait_for_locks to 1
      File-based locking selected.
      Setting global/locking_dir to /run/lock/lvm
      Setting global/prioritise_write_locks to 1
      metadata/pvmetadataignore not found in config: defaulting to n
      metadata/pvmetadatasize not found in config: defaulting to 255
      metadata/pvmetadatacopies not found in config: defaulting to 1
      Locking /run/lock/lvm/P_orphans WB
        _do_flock /run/lock/lvm/P_orphans:aux WB
        _do_flock /run/lock/lvm/P_orphans WB
        _undo_flock /run/lock/lvm/P_orphans:aux
        /dev/drbd0: Added to device cache
        Opened /dev/drbd0 RO O_DIRECT
      /dev/drbd0: size is 3886335720 sectors
        /dev/drbd0: block size is 4096 bytes
        /dev/drbd0: Skipping: Partition table signature found
        Closed /dev/drbd0
        /dev/drbd0: Skipping (cached)
      Setting devices/sysfs_scan to 1
      Setting devices/md_component_detection to 1
      Setting devices/multipath_component_detection to 1
      Setting devices/ignore_suspended_devices to 0
      Setting devices/cache_dir to /run/lvm
      Setting devices/write_cache_state to 0
        Opened /dev/drbd0 RO O_DIRECT
      /dev/drbd0: size is 3886335720 sectors
        /dev/drbd0: block size is 4096 bytes
        /dev/drbd0: Skipping: Partition table signature found
        Closed /dev/drbd0
  Device /dev/drbd0 not found (or ignored by filtering).
      Unlocking /run/lock/lvm/P_orphans
        _undo_flock /run/lock/lvm/P_orphans
 
This is your issue:
Code:
/dev/drbd0: Skipping: Partition table signature found

pvcreate is refusing to run since it has detected a partition table on the drbd volume
Is this maybe left over from a previous use of this disk?

If you are sure everything is correct and you want to delete the partition table this is how you do it:
Code:
dd if=/dev/zero of=/dev/drbd0 bs=512 count=1

Once the partition table is gone pvcreate should work
 
i have re create an test environment.
i have tested with your command and it's ok. i also tested delete partiton with fdisk. thank you :)

i have run pvcreate with -vvv. Here the output :
Code:
root@proxmox02:~# pvcreate -vvv /dev/drbd0      Setting activation/monitoring to 0
        Processing: pvcreate -vvv /dev/drbd0
        O_DIRECT will be used
      Setting global/locking_type to 1
      Setting global/wait_for_locks to 1
      File-based locking selected.
      Setting global/locking_dir to /run/lock/lvm
      Setting global/prioritise_write_locks to 1
      metadata/pvmetadataignore not found in config: defaulting to n
      metadata/pvmetadatasize not found in config: defaulting to 255
      metadata/pvmetadatacopies not found in config: defaulting to 1
      Locking /run/lock/lvm/P_orphans WB
        _do_flock /run/lock/lvm/P_orphans:aux WB
        _do_flock /run/lock/lvm/P_orphans WB
        _undo_flock /run/lock/lvm/P_orphans:aux
        /dev/drbd0: Added to device cache
        Opened /dev/drbd0 RO O_DIRECT
      /dev/drbd0: size is 3886335720 sectors
        /dev/drbd0: block size is 4096 bytes
        Closed /dev/drbd0
      /dev/drbd0: size is 3886335720 sectors
        Opened /dev/drbd0 RO O_DIRECT
        /dev/drbd0: block size is 4096 bytes
        Closed /dev/drbd0
        Using /dev/drbd0
        Opened /dev/drbd0 RO O_DIRECT
        /dev/drbd0: block size is 4096 bytes
      /dev/drbd0: No label detected
        Closed /dev/drbd0
        Opened /dev/drbd0 RW O_EXCL O_DIRECT
        Closed /dev/drbd0
      /dev/drbd0: size is 3886335720 sectors
        Opened /dev/drbd0 RO O_DIRECT
        /dev/drbd0: block size is 4096 bytes
        Closed /dev/drbd0
      /dev/drbd0: size is 3886335720 sectors
        Opened /dev/drbd0 RO O_DIRECT
        /dev/drbd0: block size is 4096 bytes
        Closed /dev/drbd0
        Opened /dev/drbd0 RO O_DIRECT
        /dev/drbd0: block size is 4096 bytes
        Closed /dev/drbd0
      /dev/drbd0: size is 3886335720 sectors
      Setting devices/data_alignment to 0
      devices/default_data_alignment not found in config: defaulting to 1
      Setting devices/md_chunk_alignment to 1
      Setting devices/data_alignment_detection to 1
      Device /dev/drbd0 queue/minimum_io_size is 4096 bytes.
      Device /dev/drbd0 queue/optimal_io_size is 0 bytes.
      /dev/drbd0: Setting PE alignment to 2048 sectors.
      Setting devices/data_alignment_offset_detection to 1
      Device /dev/drbd0 alignment_offset is 0 bytes.
      /dev/drbd0: Setting PE alignment offset to 0 sectors.
        Opened /dev/drbd0 RW O_DIRECT
        Wiping /dev/drbd0 at sector 8 length 8 sectors
        /dev/drbd0: block size is 4096 bytes
        Closed /dev/drbd0
    Set up physical volume for "/dev/drbd0" with 3886335720 available sectors
      Scanning for labels to wipe from /dev/drbd0
        Opened /dev/drbd0 RW O_DIRECT
        /dev/drbd0: block size is 4096 bytes
        Closed /dev/drbd0
    Zeroing start of device /dev/drbd0
        Opened /dev/drbd0 RW O_DIRECT
        Wiping /dev/drbd0 at sector 0 length 4 sectors
        /dev/drbd0: block size is 4096 bytes
        Closed /dev/drbd0
    Writing physical volume data to disk "/dev/drbd0"
        lvmcache: /dev/drbd0: now in VG #orphans_lvm2 (#orphans_lvm2) with 0 mdas
        Creating metadata area on /dev/drbd0 at sector 8 size 2040 sectors
        Opened /dev/drbd0 RW O_DIRECT
        /dev/drbd0: block size is 4096 bytes
        /dev/drbd0: Preparing PV label header zil3Ml-LOs6-Lz2S-2kdU-c0Bi-pny0-Ut2Ibv size 1989803888640 with da1 (2048s, 0s) mda1 (8s, 2040s)
      /dev/drbd0: Writing label to sector 1 with stored offset 32.
  Physical volume "/dev/drbd0" successfully created
      Unlocking /run/lock/lvm/P_orphans
        _undo_flock /run/lock/lvm/P_orphans
        Closed /dev/drbd0

and now the pvscan command :

Code:
root@proxmox02:~# pvscan
  Found duplicate PV zil3MlLOs6Lz2S2kdUc0Bipny0Ut2Ibv: using /dev/md/2 not /dev/drbd0
  PV /dev/md1    VG pve             lvm2 [1,73 TiB / 0    free]
  PV /dev/md/2                      lvm2 [1,81 TiB]
  Total: 2 [3,54 TiB] / in use: 1 [1,73 TiB] / in no VG: 1 [1,81 TiB]

it show "Found duplicate PV", and "using /dev/md/2 not /dev/drbd0"
It is normal ?

thank you
 
Last edited:
Hi,
you must disable (filter) /dev/md/2 in lvm.conf - like in the wiki (the physical device of course too).

Udo

BTW. do you think it's an good idea to use drbd on top of software-raid? DRBD is an network-raid1 so perhaps you can use one laye less between virtulization and storage?
 
Last edited:
Hi Udo,
Thank you for this. I haven't think about that and understand this filter.

About drbd on to of software-raid,
I have an installation with hardware raid and drbd and HA. It's ok.

But for this another site, i have two server without hardware raid. So i would like to test it.
There is few person, 15. So, performance isn't my first goal. I would to have HA too. I will test it first.

But i agree with you. Another layer.

A little off topic, about write cache.
If could you confirm me something.

if I understand correctly, i must use directsync or write through with drbd. Write through seems to be more safe.
no cache is advisable with hardware raid with BBU.

with software raid and drbd, must i use write through or direct sync ?
must i disable cache in VM ? (it's windows guest).

thank you.
 
Hi Udo,
Thank you for this. I haven't think about that and understand this filter.

About drbd on to of software-raid,
I have an installation with hardware raid and drbd and HA. It's ok.

But for this another site, i have two server without hardware raid. So i would like to test it.
There is few person, 15. So, performance isn't my first goal. I would to have HA too. I will test it first.

But i agree with you. Another layer.

A little off topic, about write cache.
If could you confirm me something.

if I understand correctly, i must use directsync or write through with drbd. Write through seems to be more safe.
no cache is advisable with hardware raid with BBU.

with software raid and drbd, must i use write through or direct sync ?
must i disable cache in VM ? (it's windows guest).

thank you.

If you don't have a Hardware RAID with BBU (Backup Battery Unit), for security reasons and gain a better speed for your VMs, use writethrough (you don't lose information if the node crashes, and you will have a buffer of blocks of writes in the physical host using only his free memory) and also use DRBD without cache enabled (remember that each layer of disk access may have his own cache independent)
 
ok, thank you for your remark cesarpk.

Your welcome, and only for be more clear, always that the security of data is the priority, you must use only a of these two options:

Directsync: never do cache, only writes in the platters of the disk, and generally used for SAN, iSCSI, etc of high speed communications. In several cases also can be used with writethrough in this scenery.

Writethrough: writes in platters of the disk and also do cache of his writes in the RAM free of physical host, and generally used for local storages. So if the host have much RAM, more write cache will have your physical host and therefore, better performance proportional to amount of RAM free.

RAID controller:
Always that it have cache with BBU enabled, enable the write cache in the RAID controller, the performance is fantastic (for much difference), and the RAID controller will deceive to the VMs that believe that the data have been stored in the platters of disk. In this scenery, thanks to cache and the BBU, you will don't have worry for lost of data if the host hang or breaks (note: a new BBU can endure 72 Hrs, ie until the battery runs out.

DRBD:
The documentation of DRBD tell us how take advantage of the chache of the RAID controller for gain speed, see the documentatation of this topic (see tuning of DRBD).
Also the documentation tell us how configure DRBD if you don't have a RAID controller with BBU for avoid the lose of data, that is the same case for mdadm (see tuning of DRBD)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!