How to safely enable KRBD in a 5 node production environment running 7.4.19

J

john.gaspode

New Member
Oct 8, 2024
3
1
3
I’m running a 5-node Proxmox cluster on version 7.4.19 with two storage pools: one on spindle drives and the other on SSDs. Both pools host live VMs, but performance is slower than expected, with high IOWait states.

I’ve read that enabling KRBD can improve performance, but I haven’t found clear guidance on how to safely enable it.

Currently, I’m in the process of updating the cluster to Proxmox 8 but have paused after upgrading all nodes to version 7.4.19. The Proxmox OS itself is installed on spindle drives.

Here are my questions:

What happens when I enable the KRBD option for the pools?
Will live VMs need to be restarted to apply the changes?
Do the host machines require a restart after enabling KRBD?
Should I shut down all VMs before enabling the option?
Would it be better to complete the upgrade to Proxmox 8 before enabling KRBD?

Thank you for your help!
 
Hello,

What happens when I enable the KRBD option for the pools?
Click to expand...
You will use the RBD kernel module instead of librbd. In certain setup/workloads KRBD could offer better performance.

Will live VMs need to be restarted to apply the changes?
Click to expand...
It depends on what do you mean by a reboot. To be sure, I would recommend to *completely* shut down the VM and start it again.

Do the host machines require a restart after enabling KRBD?
Click to expand...
No.

Should I shut down all VMs before enabling the option?
Click to expand...
No.

Would it be better to complete the upgrade to Proxmox 8 before enabling KRBD?
Click to expand...
It should not really matter whether your upgrade for the purpose of enabling KRBD. However, Proxmox VE 7 is End-of-Life [1] so yes, I would recommend to update it regardless.

You can go to your storage settings at Datacenter->Storage->{RBD storage} and check the KRBD checkbox in the dialog to enable it. As mentioned you will need to cold-reboot the guests for it to take into effect.

[1] https://forum.proxmox.com/threads/proxmox-ve-support-lifecycle.35755/
 
Hi,
Thanks @john.gaspode for the question, it was a topic for me last week. I had no urgency so for the moment I hadn't done anything until I found the info ^^

Instead of cold-reboot, if we live migrate VMs to another host (and eventually migrate back after) will it be sufficient ?
 
Hi @tomatoschewps,

Sorry for the late reply I was away over Christmas.

I'm no expert but I don't think migrating the VM from one host to another will be sufficient.

We rebooted all the VM's in our network.
 
Yes, migrating the VM to another host and back to the host with the newly enabled KRBD setting will start a new QEMU instance for the VM using KRBD. To check if a VM is using KRBD, these grep will match in each case:

librbd:
Code: 
qm showcmd VMID | grep --color "\-drive 'file=rbd:"

KRBD:
Code: 
qm showcmd VMID | grep --color "\-drive 'file=/dev/rbd"

If you have different Ceph pools you may have KRBD enabled in one and not in another, so you can place VM disks on different pools depending on you needs.
 
  • Like
Reactions: gurubert and tomatoschewps
VictorSTS said:
Yes, migrating the VM to another host and back to the host with the newly enabled KRBD setting will start a new QEMU instance for the VM using KRBD. To check if a VM is using KRBD, these grep will match in each case:

librbd:
Code: 
qm showcmd VMID | grep --color "\-drive 'file=rbd:"

KRBD:
Code: 
qm showcmd VMID | grep --color "\-drive 'file=/dev/rbd"

If you have different Ceph pools you may have KRBD enabled in one and not in another, so you can place VM disks on different pools depending on you needs.
Click to expand...
A great way to check whether it is already running via the KRBD, thank you
 
You must log in or register to reply here.
Top Bottom