[SOLVED] Advice for storing database

[nivxomxorp]

Hi everyone,

I would like to use the snapshot feature only for system files of my VM and not for database files since I dont want to restore previous data if I do a rollback.

What is the best solution for a disk with database files and also MS Office documents ?

Vincent

 

[wolfgang]

Hi,

generally it is recommended to install the qemu-guest-agent,
because it will freeze the fs inside the vm before it takes a snapshot.

 

[nivxomxorp]

Hi,

Thanks for the information but that's not my question : is it possible to disable the snapshot for a disk (which is containing the database) but enable for the system disk, so in case of system failure, restoration of the snapshot while not restore an old version of the database.

Regards
Vincent

 

[wolfgang]

No this is not possible and makes no sense, because in the worse case you would corrupt your db and files.
The only thing what you can do is deactivate the backup on this disk.

 

[hansm]

No this is not possible and makes no sense, because in the worse case you would corrupt your db and files.
The only thing what you can do is deactivate the backup on this disk.

IMO it makes sense. I would like to use it too, but for a slightly different situation. For a database server I use a system disk including all databases and want to use PVE snapshot function for this disk. The second disk is for GlusterFS, shared with another server. Imagine vm01 and vm02, both have a second disk for GlusterFS. I really don't like to add this disk in a snapshot, it will definitely corrupt GlusterFS when I need to rollback a snapshot on one of the servers. Corrupting it or going into split-brain, or something like that, we need to avoid that. Besides that I share the opinion of TS because you probably want to rollback your failed system update but it makes no sense to rollback your database data because only the OS is corrupted and not your databases.

 

[dcsapak]

IMO it makes sense. I would like to use it too, but for a slightly different situation. For a database server I use a system disk including all databases and want to use PVE snapshot function for this disk. The second disk is for GlusterFS, shared with another server. Imagine vm01 and vm02, both have a second disk for GlusterFS. I really don't like to add this disk in a snapshot, it will definitely corrupt GlusterFS when I need to rollback a snapshot on one of the servers. Corrupting it or going into split-brain, or something like that, we need to avoid that. Besides that I share the opinion of TS because you probably want to rollback your failed system update but it makes no sense to rollback your database data because only the OS is corrupted and not your databases.

but this can also not work directly...
if you do snapshot with ram (without ram its a bit different), the vm holds info about this disk in ram (e.g. filesystem structure, in your case the gluster process will have certain infos about the bricks in ram)

now if you would then rollback this snapshot, the vm will have an inconsistent view of the disk, which will likely also corrupt your disk..

 

[fabian]

but this can also not work directly...
if you do snapshot with ram (without ram its a bit different), the vm holds info about this disk in ram (e.g. filesystem structure, in your case the gluster process will have certain infos about the bricks in ram)

now if you would then rollback this snapshot, the vm will have an inconsistent view of the disk, which will likely also corrupt your disk..

we could catch this though? snapshot with RAM is only possible if all the disks/volumes are included in the snapshot.. a similar feature is often requested for containers, where it would make sense to exclude certain mount points from snapshotting (e.g., big NFS shares bind mounted in lots of containers).

 

[hansm]

but this can also not work directly...
if you do snapshot with ram (without ram its a bit different), the vm holds info about this disk in ram (e.g. filesystem structure, in your case the gluster process will have certain infos about the bricks in ram)

now if you would then rollback this snapshot, the vm will have an inconsistent view of the disk, which will likely also corrupt your disk..

You're possibly right, I'm not sure about that, I think RAM will only serve as read cache and will not corrupt the data, at most it returns objects from RAM (cache) that isn't up-to-date but afaik Gluster checks the directory structure on both ends on every listing and will update it's RAM accordingly. Let's assume you're correct and it would corrupt data on the Gluster share, we could prevent that by snapshotting with RAM, but rollback with network disconnected? I haven't tested this, is it possible? Than after the rollback you can restart Gluster and/or cleaning caches, or even reboot to be sure. Snapshotting the disk containing the data of the Gluster share won't give me any benefit, it takes resources to snapshot and store it but I will never use it. The time it takes to take a snapshot is way shorter when I could skip the additional disk (Gluster for me, database for TS). My Gluster share can grow up to hundreds GB's or even further.

we could catch this though? snapshot with RAM is only possible if all the disks/volumes are included in the snapshot.. a similar feature is often requested for containers, where it would make sense to exclude certain mount points from snapshotting (e.g., big NFS shares bind mounted in lots of containers).

I like this idea! Maybe the option to include RAM should be disabled by default when not all disks are selected, but when you tick the box to include RAM a warning will pop up about the risk of including RAM in a snapshot when not all disks are going to be snapshotted. It will give us the possibility to do it if we really really want it and know what we're doing.