Hi everyone, I'm here because I think I'm missing something important.I have read and watched many guides and videos, but I can't get the HA to work when I "kill" one of the two nodes.The quorum remains respected, I set the number of replies to the minimum. Yet I get an error that I can't interpret.Maybe I'm just getting the basics wrong?Thank you
HA cluster with two node and qDevice + CEPH: don't work HA: why?
- Thread starter cybermod
- Start date
Letting us know what error that is would be helpful.
How many Ceph MONs do you have? Because Ceph needs to also have Quorum on the MONs (similar to PVE), you need a majority of MONs present. In a 2-node cluster.... One of the reasons why Ceph is only useful once you have at least 3 full nodes!
dear, tnx for your reply!
my configurations is:
2 node + qDevice on Ubuntu Server 22.04.
I have two monitor.....
So.... i can't use CEPH for HA because i have not 3 full node but only 2 full node + 1 qDevice....
i'm confused....
what is the point of having qdevice?
Can i have HA on 2 nodes?
maybe...i need a shared external storage (like nas) and like vmware.
my configurations is:
2 node + qDevice on Ubuntu Server 22.04.
I have two monitor.....
So.... i can't use CEPH for HA because i have not 3 full node but only 2 full node + 1 qDevice....
i'm confused....
what is the point of having qdevice?
Can i have HA on 2 nodes?
maybe...i need a shared external storage (like nas) and like vmware.
TASK ERROR: start failed: command '/usr/bin/kvm -id 100 -name 'ubusrvHAlabtest,debug-threads=on' -no-shutdown -chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server=on,wait=off' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/100.pid -daemonize -smbios 'type=1,uuid=f0349f09-e1c4-47ef-a938-5c15b2464175' -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vnc 'unix:/var/run/qemu-server/100.vnc,password=on' -cpu qemu64,+aes,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+pni,+popcnt,+sse4.1,+sse4.2,+ssse3 -m 2048 -object 'iothread,id=iothread-virtioscsi0' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' -device 'vmgenid,guid=39c0cf2e-d3c2-413a-84f4-cfee196311df' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'VGA,id=vga,bus=pci.0,addr=0x2' -chardev 'socket,path=/var/run/qemu-server/100.qga,server=on,wait=off,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:8cdf10601a72' -drive 'if=none,id=drive-ide2,media=cdrom,aio=io_uring' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=101' -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1,iothread=iothread-virtioscsi0' -drive 'file=rbd:ceph01/vm-100-disk-0:conf=/etc/pve/ceph.conf:id=admin:keyring=/etc/pve/priv/ceph/ceph01.keyring,if=none,id=drive-scsi0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=BC:24:11:9C:33:41,netdev=net0,bus=pci.0,addr=0x12,id=net0,rx_queue_size=1024,tx_queue_size=256,bootindex=102' -machine 'type=pc+pve0'' failed: got timeout
Last edited:
You need a quorum for the PVE cluster, this where you can use a qdevice.
Ceph needs a separate, its own quorum. The Ceph monitors are the voters in this case.
So currently you've got a quorate PVE cluster (2/3), but Ceph is without quorum (1/2).
sure, but not with Ceph. Either use a network share, or alternatively local ZFS + the guest replication feature. The downside of the ZFS + Replication approach is, that it is async. In a worst case scenario, you might lose some data (since the last successful replication). Though it is possible to run the replication as short as each minute.
tnx, i tried it and now i feel myselfe moore confidence.
tnx for your clarify me
really appreciate