Backup of OCFS2 filesystem

C

cybertinus

New Member
Jan 2, 2024
9
1
3
The Netherlands
Hello all,

I'm running a cluster of servers for one of my customers, which are Debian 12 based VMs on my PVE cluster. 2 of those servers are web servers, and need to have exactly the same content to be served by the web servers. I'm using DRBD to sync the data in real time between the two servers. On top of the DRBD block device I'm running OCFS2, to have a cluster aware filesystem.
This all runs file and the data is instantly available on both servers, like designed!

The issue I'm having is in regards to backup. These VMs are backed up to a Proxmox Backup Server. This gives me the advantage that I can restore single files from the backups. Except for this OCFS2 filesystem. It looks like it isn't supported by PBS or PVE, not sure.

A few questions:
1. Do I even have valid backups now? Can I at least restore the complete filesystem, if needed?
2. Can I get single file restoration to work again by installing OCFS2 filesystem software (enable kernel drivers and such) on PBS? Or do I need to install this on my PVEs? Or both?

Or should I forget about getting files out of the OCFS2 filesystem, and do I need to think about a different way to get single file backups again.
 
Hi @cybertinus ,

How is the data presented to the VMs? Do they see QEMU disks with filesystem/data that is being replicated in the background? Are these QEMU disks backed by QCOW files located on OCFS?

If the answer is "yes", then from PBS perspective it does not matter that the underlying FS is OCFS. The backup is done at a higher QEMU level.
The best way to ensure that you have a good backup is to do a periodic restore. Have you tried it?

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
Hello @bbgeek17,

I'm running the integrated Ceph within my proxmox cluster. For the VMs I created an extra harddisk, on my Ceph cluster. And then I run DRBD and OCFS2 inside the VMs. So, the VM has an extra block devices, and I run DRBD and OCFS2 on top of that.
So, I'm not using qcow anyware.
 
Both PBS and PVE's built-in backup process operate at the QEMU disk level, not at the VM file level. The VM disk data is backed up and stored as a collection of indexed and hashed file chunks.

During single-file-recover, these chunks are read and mounted as a virtual disk.

In your case, the Ceph disk is presented to the VM as a raw QEMU block device. However, as you described, there are at least two additional layers - DRBD and OCFS - before PBS can access files. You’ll need to ensure that PBS has the necessary drivers, modules, and tools to interpret these layers.

If the required software is available, it should be automatically loaded based on the headers found on the virtual disk.

Since your environment is already running, have you attempted a restore yet?

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: Johannes S
OK, Thanks for your answer. I will install DRBD and OCFS2 on my PBS machine, and see if I can enter the files, which I highly doubt, still.

I just tried to restore an entire VM and tried to access the data inside the DRBD/OCFS2 construction. And this worked! Which is cool! So, I have at least a valid backup, in case the VM crashes completely. I did have to force the DRBD to be online though, after the restore. I disconnected the NICs in the VM, to not interfere with the production VMs, so DRBD couldn't connect to it's partner on the other node.
This is why I think that it isn't possible to restore files from PBS directly, because PBS can't force DRBD online. But: I will test with it and report back.
 
cybertinus said:
This is why I think that it isn't possible to restore files from PBS directly, because PBS can't force DRBD online. But: I will test with it and report back.
Click to expand...
Sounds like file level restore from PVE gui is a sacrifice you have to make in your particular environment. You have a 3rd party storage design requirement for manual intervention to bring that storage online.

You could experiment with hooks and report back whether it helped.

Good luck

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
if you want file level restore, you could additionally run "host" backups using proxmox-backup-client inside the VM. the file-restore-VM for regular VM backups will likely not support recovering such complicated storage setups for the foreseeable future. we do plan on making it easier to do custom file restore/easier "live backup access" in someway, once that lands you could probably automate it yourself though ;)
 
You must log in or register to reply here.
Top Bottom