Proxmox Backups - An option to not use HCP Snapshots
- Thread starter marotori
- Start date
I would be happy with that too - my concern is the problem creeping in again. If you search the net - you will find that this has been an ongoing issue in the redhat kernel for a very long time.
Sent from my iPhone using Tapatalk
it's true, but it would be better to support hcp in vzdump, at least until the lvm issue will be fixed (who knows when), as an alternative to the minority who have the lvm issue.
without this pve is a product that sometimes crashes on some hardware.
i know it's beta but you want to release the final one time, and you don't want to leave us without a working solution.
Simply claiming that something is better is easy. But I would prefer hard numbers - so please provide a benchmark or test case that shows that HCP is better (or reproduces the LVM crash). We can then try to debug an make things better.
Last edited:
Maybe I should log one then 
.
To date I have been using these forums to post the issue - guess that is the wrong location.
Overall - lvm is stable. However... I have 17 servers all different specs.
6 of them crash. The rest don't.
I am certain that this is related to disk io / driver issues. But to date; no 100% fix has been found. As a result - i have looked at hcp to solve the problem. This works perfectly
No hangs, lookups or general crashes when backing up.
I am happy to do the coding? Really quite a simple job to include the function.
Rob
Sent from my iPhone using Tapatalk
. To date I have been using these forums to post the issue - guess that is the wrong location.
Overall - lvm is stable. However... I have 17 servers all different specs.
6 of them crash. The rest don't.
I am certain that this is related to disk io / driver issues. But to date; no 100% fix has been found. As a result - i have looked at hcp to solve the problem. This works perfectly
No hangs, lookups or general crashes when backing up.
I am happy to do the coding? Really quite a simple job to include the function.
Rob
Sent from my iPhone using Tapatalk
I can reproduce the lvm crash on my systems easily. Simply generate high io... Do a 'very long' backup. And the unmount the snapshot.
The system goes into a state where all disk io stops. Only solution is a reboot.
I am not saying hcp is better. And don't want to get into a flaming war over the merits. I am simply saying that hcp is a suitable workaround for users like myself who have the lvm crashes.
Maybe not the 'best fix'. Fixing lvm would be nice - but it is a fix that is easy.. And provides choice
Sent from my iPhone using Tapatalk
Maybe I should log one then
To date I have been using these forums to post the issue - guess that is the wrong location.
Overall - lvm is stable. However... I have 17 servers all different specs.
6 of them crash. The rest don't.
I am certain that this is related to disk io / driver issues. But to date; no 100% fix has been found. As a result - i have looked at hcp to solve the problem. This works perfectly
No hangs, lookups or general crashes when backing up.
As far as I followed your issue most are related to lsi raid cards with the latest 2.0 kernel - is this true?
we don´t think that its a good idea to include non-free software in a free software product, this makes no sense for me. btw, instead of lvm snapshot there will be also qemu live copy or even btrfs sound promising.
additionally, if you need live backup solution and vzdump lvm snapshot does not work in your setup there are already a lot of non-free backup tools.
It works with ext3/4 xfs etc... Most Linux file systems
I also don't see how that can work for block devices we pass directly to KVM. IMHO that will not work or is at least very dangerous.
I agree. Most issues seem to source on Lsi cards. But strangely.. This not appear to cause problems with kernels pre 2.6.32.
Redhats answer to the problem was - run an earlier kernel! Not really a solution.
I agree about not providing a non gpl product in a gpl solution. You will note that I have always said - do not include it. However... Adding an option to use the product is a good compromise.
At lest
At least an option is available to do a safe vzdump?
Sent from my iPhone using Tapatalk
Redhats answer to the problem was - run an earlier kernel! Not really a solution.
I agree about not providing a non gpl product in a gpl solution. You will note that I have always said - do not include it. However... Adding an option to use the product is a good compromise.
At lest
At least an option is available to do a safe vzdump?
Sent from my iPhone using Tapatalk
Last edited:
from hcp homepage:
- Works on any server-based file system or block device
- Install without rebooting
- Eliminates LVM pre-requisite
- Requires no pre-allocation or pre-configuration of storage
- Uses existing free space on your disk to maintain snapshots
- Outstanding performance compared to LVM snapshots
- Works on most High-Availability and Shared Storage
- Specialized performance awareness for: Ext2, Ext3, Ext4, and Reiserfs3
- Snapshots are readable and writable
- Keep multiple snapshots of each disk or volume
I totally get the point you guys may have in keeping to lvm - and reluctance to use alternative systems.
Remember.... This is all about choice. No solution is a perfect fit. We all do things a bit differently.
The beauty of open-source is that it provides the ability for alternative ideas and solutions (work a rounds) to be presented and used.
Rob
Sent from my iPhone using Tapatalk
Remember.... This is all about choice. No solution is a perfect fit. We all do things a bit differently.
The beauty of open-source is that it provides the ability for alternative ideas and solutions (work a rounds) to be presented and used.
Rob
Sent from my iPhone using Tapatalk
no, you are just wrong. we suggested that you help to fix the issue in LVM. additonally I told you two alternative solutions for the future.
also no. the beauty of Proxmox VE is NOT to include and develop all possible ways. if we do this we would end up with a completely different Proxmox VE with no success.
we try to choose the best solutions. if you want all options, you can still install whatever you want but we cannot include this as its not compatible with our license model. so you are free to install what you want with our software as long as you respect the license but you cannot force us to do what you suggest.
if you want to convince us provide benchmarks instead of telling us that we are reluctant to alternatives. and you still ignore the licensing issue, making it impossble to include it - or do you plan to pay the OEM fee for 30.000 installations (and counting
- I doubt that you are prepared for this but if yes, contact our sales team.Why oh why does written text get miss-interpretted!
I feel you have taken my comments as negative or critisisms - yet that is not what I have intended.
I fully understand that the point is to make the software the best; and that you need to act as gate keepers to ensure the end product works - follows key semantics - and is the best it can be. That is essential and a job that needs to be done.
To clarify a few things:
- I have not raised this thread to try say that hcp is better than lvm. I personally dont care what the solution is - as long as we find a solution.
- I am not an idiot end user. I am a competant developer & sysadmin. Quite capable of solving this.
- I am not simply expecting you to jump and fix things that work fine for you. I think you have a great product; and would actually like to plug some time in to helping improve it.
- I have raised the point that there is an issue with LVM snapshots not working well across all systems. From my point this is a big issue as it kills off a very effecting backup system. (I normally do vzdumps weekly.. and bacula daily)
- I do not think there is any merrit in a benchmark of performance of HCP. This is a waste of time. The original post and my questions does not in anyway raise the question of comparing the two solutions.
- I would love a fix to lvm as this would solve all problems; and save me having to install third party software. I have however spent the past 2 weeks hacking away and have not got a clean fix.
What I have asked originally is:
- is is possible to get an option added to vzdump to allow hcp backups as an alternative to lvm?
- If not; no problem. I will write my own alternative to vzdump "hcpdump' to allow my systems to use this for backups.
- If you are interested; will you accept a patch written by myself?
Rob
I feel you have taken my comments as negative or critisisms - yet that is not what I have intended.
I fully understand that the point is to make the software the best; and that you need to act as gate keepers to ensure the end product works - follows key semantics - and is the best it can be. That is essential and a job that needs to be done.
To clarify a few things:
- I have not raised this thread to try say that hcp is better than lvm. I personally dont care what the solution is - as long as we find a solution.
- I am not an idiot end user. I am a competant developer & sysadmin. Quite capable of solving this.
- I am not simply expecting you to jump and fix things that work fine for you. I think you have a great product; and would actually like to plug some time in to helping improve it.
- I have raised the point that there is an issue with LVM snapshots not working well across all systems. From my point this is a big issue as it kills off a very effecting backup system. (I normally do vzdumps weekly.. and bacula daily)
- I do not think there is any merrit in a benchmark of performance of HCP. This is a waste of time. The original post and my questions does not in anyway raise the question of comparing the two solutions.
- I would love a fix to lvm as this would solve all problems; and save me having to install third party software. I have however spent the past 2 weeks hacking away and have not got a clean fix.
What I have asked originally is:
- is is possible to get an option added to vzdump to allow hcp backups as an alternative to lvm?
- If not; no problem. I will write my own alternative to vzdump "hcpdump' to allow my systems to use this for backups.
- If you are interested; will you accept a patch written by myself?
Rob
We simply state our opinion, and there is no negative attitude here (sorry if you feel that way - that was not our intention).
You did not provide any details, and there is no reproducible test case - so it is really hard to work on a fix.
No, because we do not want to support that. Instead, we want our users to use open-source (LVM).
So we are interested in a test-case which shows the problem.
I agree. It cannot replace LVM for all core functions.
I personally have not experienced any issues with LVM using these sort of functions. The issues only occur when doing snapshot backups. That is why I have only suggested it for an option in the vzdump function
Rob
I personally have not experienced any issues with LVM using these sort of functions. The issues only occur when doing snapshot backups. That is why I have only suggested it for an option in the vzdump function
Rob
Right...
Case scenarios:
These are difficult to debug - I aggree. If the issue does not occor on your machine - then how do you fix it!
Here are some examples:
http://www.redhat.com/archives/linux-lvm/2010-April/msg00035.html
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/605551
http://harvey.nu/lvm_snapshot_lvremove_hang.html
http://osdir.com/ml/linux-lvm/2010-04/msg00073.html
You will note the symtoms are all slighltly different - making debug rather hard!
In my case.. I can reproduce the issue consistantly by creating a snapshot. Leaving it open for 12-24 hours - then trying to remove it. (A typical vzdump of a rather large container)
No idea if this hels try fix it?
p.s. I am beginging to see your thoughts on HCP. If you have a kvm machine using an lvm volume direct.. I guess there is no way the backup using vzdump could work?
Rob
Case scenarios:
These are difficult to debug - I aggree. If the issue does not occor on your machine - then how do you fix it!
Here are some examples:
http://www.redhat.com/archives/linux-lvm/2010-April/msg00035.html
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/605551
http://harvey.nu/lvm_snapshot_lvremove_hang.html
http://osdir.com/ml/linux-lvm/2010-04/msg00073.html
You will note the symtoms are all slighltly different - making debug rather hard!
In my case.. I can reproduce the issue consistantly by creating a snapshot. Leaving it open for 12-24 hours - then trying to remove it. (A typical vzdump of a rather large container)
No idea if this hels try fix it?
p.s. I am beginging to see your thoughts on HCP. If you have a kvm machine using an lvm volume direct.. I guess there is no way the backup using vzdump could work?
Rob