Survey: Proxmox VE Kernel with or without OpenVZ?

[mangoo]

I wonder how lacking a proxmox version with lxc and kvm would be in comparison to, say the current 1.4?

I guess implementing lxc would be welcome here, but would need the following, at the minimum:

- more manpower to support a third virtualization technology,
- yet more manpower to test it, since lxc is not yet widely used,
- support both OpenVZ and lxc at the same time, since it's not yet clear how painless migration from one to another would be.


You are free to contribute of course

 

[danhs]

I agree with mangoo.

KVM is in the mainline kernel. That's a big deal. Maintaining a patched kernel solely for OpenVZ that does not benefit from the security patches, and updates of the mainline kernel/Debian team seems like a lot of extra work to bite off. Work which can't be maintained indefinitely.

I understand the rationale for maintaining support for OpenVZ, but let's face facts and recognize the costs involved.

• Kernel issues as noted above.
• There are some major issues with continuing to try to patch kernel. Take a look at the recent virtio_net issues for KVM. This is not trivial. Having virtio NIC fail is a pretty big issue. This has only become an issue because of the effort to reconcile legacy OpenVZ with KVM and new kernel advancement. If it weren't for OpenVZ proxmox could run off of a typical debian kernel and not have these KVM kernel patch issues.
• OpenVZ is obviously losing development steam and support. That's why it can't keep up. Sticking with legacy applications and letting proxmox be dragged back by openvz helps no one.
• A lot of the rationale I'm reading for OpenVZ: performance, doesn't seem to actually be true. See some benchmarks at link provided http://www.scribd.com/doc/4916478/comparison-of-open-source-virtualization-technology also note mangoo's point about KMS which will allow KVM to use less memory.
• Although the benchmark comparison link I provided is a bit old (Aug. 2008) it's instructive. Firstly, OpenVZ has lost development steam, therefore, I doubt the performance of OpenVZ is dramatically better than it was in Aug 2008. Secondly, the biggest area where KVM is deficient compared to OpenVZ is IO. If you use the virtio drivers for KVM this is not an issue. Thirdly, KVM is the beneficiary of a *lot* of development resources (money, and skilled developers). At the time of those benchmarks, KVM was better than (or at the very least, comparable) to OpenVZ. If you consider that KVM is improving very rapidly (the opposite of OpenVZ) and then consider that KVM would have performed better with the benefit of virtio, the reality becomes clear.

I can understand why containers would be very appealing to many people. It makes a lot of sense. However, it doesn't seem like OpenVZ is the right container technology to choose going forward.

LXC appears to be a better emerging alternative. Apparently, it's already in the mainline kernel, IBM is investing in its active development, and some people are looking into whether existing mature solutions like OpenVZ can use the new LXC kernel functionalities.

It seems that pushing OpenVZ to adapt to utilize LXC is a much better solution than holding up proxmox development.

Proxmox is a great bare metal platform for virtualization management. I think it's the best available; almost certainly the best free platform. And if you consider bang-for-buck it beats the competition by miles. It's getting better with each release.

However, legacy applications shouldn't slow it down. It would be a shame if legacy, half-dead projects which don't receive adequate development attention stall proxmox.

Likewise, it would be a shame if proxmox gets mired in maintaining code that is *way* outside of its sweet spot (ie if proxmox developers are forced to maintain kernel patches and back patches and the like instead of benefitting from upstream work).

If you *really* need openvz you should use it! Stick with an old version of proxmox. No one forces you to upgrade. However, you should realize, OpenVZ is not improving quickly, and proxmox's advancement shouldn't be forced to similarly stall advancement because of OpenVZ.

Simple solution: use an old version of Proxmox.

 

[adam0x54]

+1 for latest kernel and kvm only.

 

[iti-asi]

The biggest advantage of Proxmox VE for us is the really nice web frontend for both OpenVZ and KVM.

We use OpenVZ a lot more than KVM, and are slowly migrating our big Linux-VServer setup to PVE, but some of the servers are also hosting Windows boxes which were migrated from old hardware servers to KVM images.

Dropping OpenVZ would mean we'd have to look for something else.

 

[JedMeister]

+1 for retaining OpenVZ (alongside KVM).

For me this is a complete no brainer. I only use KVM for Win VMs (and I only have 1 of those presently although have done some testing in the past). I don't care what benchmarks may say in the real world (at least on my server) the performance advantage of OpenVZ is clear (using Linux VMs obviously). Also I find it handy being able to see at a glance (from the VM page in Proxmox VE Admin interface) the amount of disk space being used. In fairness to KVM, this comparrison is only within Proxmox 1.3 so perhaps the improvements in later versions make it more competitive?

 

[MeneM]

I've been thinking about this for a while now.

And if only KVM would be as frugal or memory savvy as Openvz is, I would not mind completely migrating to KVM.

I like openvz because it is low on resources. Not that much memory needed for a bare bones container.

The KVM approach needs the amount of memory in total as you assigned it to have. Same goes for the hard drive I guess. But that's less of an issue.

Thanks,
MeneM

 

[danhs]

Let me address your main points:

Ignore benchmarks

I don't care what benchmarks may say in the real world (at least on my server) the performance advantage of OpenVZ is clear (using Linux VMs obviously).

While I can appreciate that you "feel" openvz is better, citing feelings alone is not an argument which is compatible with any sort of reasonable (technical) debate or discussion. If we can't agree on objective standards of what is "better" and what is "worse" then we can't have any sort of meaningful debate. Benchmarks seem like a good metric to me.

If the only metric that matters is your subjective opinion, then no matter what happens, I can't win if I have an opinion that is different than yours. Perhaps those benchmarks aren't good ones. If you have better benchmarks, then please use them. We can debate what a good objective metric is: you can measure "good" in any number of ways.

However, if the only evidence you can present is "it feels better" then I can't possibly demonstrate that an alternative is better if you're not in the mood to agree with that position. So, I have to fundamentally reject this position.


Disk usage

Also I find it handy being able to see at a glance (from the VM page in Proxmox VE Admin interface) the amount of disk space being used.

This is a fair point. But, perhaps this can be addressed in the near future. This blog provides a good 10 second introduction to libguestfs. Take a look at the official webpage or the wikipedia page.

It looks like libguestfs can address your needs plus do a whole lot more in terms of maintenance of virtual machines: backup, configuration maintenance, cloning, etc...it actually seems very powerful.


Old version
Lastly, I haven't yet heard a response to my original proposal. If you're satisfied with openvz, and it is working well for you, why do you have a need to upgrade to newer proxmox?

What new features are you looking for, or would you expect from a new version?
Openvz developers don't seem interested in keeping pace with Proxmox as noted by Martin in this interview. Therefore, no new features for proxmox which enhance openvz *can possibly* be developed. So, from a purely openvz perspective, what would a new release accomplish? If the answer is "nothing", then why are you interested in new versions of proxmox?


What would be the plan if keeping old kernel were maintained?
Lastly, there's the big question I raised regarding how proxmox development would need to progress to maintain legacy openvz, without the intention of adding new features as that is impossible.

Would it go something like this?:

• Kernel plan:
  • Option A: Backport newer kernel features, updates, security fixes into old openvz kernel.
    • This implies no new hardware or KVM features, or whatever other new features are added to the kernel. Also, it means that proxmox will become progressively more out of step with debian mainline as they progress further and further.
  • Or: assume traditional role of openvz project and try to modify mainline kernel so it works with openvz. This seems like a big task.
• Either way, proxmox developers now step into the role of long term kernel modifiers. Possibly something like distribution modifers as they try to maintain a stable distribution platform when Squeeze and other debian releases come out.

I don't think that plan makes sense for a project that is trying to progress, become more powerful, and add new features. Proxmox would become mired in maintenance work which is better handled upstream.

 

[dietmar]

While I can appreciate that you "feel" openvz is better, citing feelings alone is not an argument which is compatible with any sort of reasonable (technical) debate or discussion. If we can't agree on objective standards of what is "better" and what is "worse" then we can't have any sort of meaningful debate. Benchmarks seem like a good metric to me.

Unfortunately, making objective benchmarks is quite difficult. For example, when you compare Disk speed you need to make sure that you test the same region of your hardisk, ... In short, most benchmarks are simply wrong and not reproducable.

 

[dietmar]

Either way, proxmox developers now step into the role of long term kernel modifiers.

Yes, this is quite bad - we waste a lot of resources for that.

 

[danhs]

Unfortunately, making objective benchmarks is quite difficult. For example, when you compare Disk speed you need to make sure that you test the same region of your hardisk, ... In short, most benchmarks are simply wrong and not reproducable.

Fair enough. Benchmarks are very often inherently flawed, and creating completely fair, objective ones is challenging.

I don't want to get too caught up in this point because a debate about the merits of benchmarks distracts from the larger point about the viability of openvz with regards to proxmox. That said, I don't believe JedMeister's assertion that openvz performs better than KVM is well supported by the statement "in the real world (at least on my server) the performance advantage of OpenVZ is clear".

If the performance advantage is so clear, it should be very easy for him to offer objective metrics/indicators of openvz's superiority. Are tasks completing faster? Do you wait less time for a server response? Of course there will be external events that influence performance (or perception of performance), such as more users, non-optimal configurations, etc..
But, if it is *so* clear that this is his strong preference, I'd expect that there'd be at least *some* objective evidence.

You're right: perfect, unbiased benchmarks may not be possible. But, that doesn't mean that in lieu of perfect benchmarks we should accept unsubstantiated personal feelings as a subsitute.

Daniel

 

[dietmar]

You're right: perfect, unbiased benchmarks may not be possible. But, that doesn't mean that in lieu of perfect benchmarks we should accept unsubstantiated personal feelings as a subsitute.

Sure, I have to agree here. Besides, performance is not the only point - usability (feeling) is also a meassure. Almost any unix admin I know likes to work with leightweight containers (its just a feeling thought).

 

[Didelo]

Well, we actually did IO benchmarks when we were deciding between VMware ESXi 3.5, Citrix Xenserver and OpenVZ. For good measure we also included KVM.

I don't have the numbers anymore, but OpenVZ beats all of them hands down: it was almost as fast as a bare installation of CentOS (the OS we used for testing inside the virtual machines). KVM followed by quite a margin (think around 60% of CentOS bare metal performance), shortly followed by Citrix XenServer, and the list was closed by VMware ESXi.

Be aware, this was on our Dell servers. Your results may be different with other hardware. But it does make a lot of sense that OpenVZ will always be faster, because OpenVZ IS almost a bare metal Linux installation and all the other ones use real hypervisors. And IO is always difficult for hypervisors, which is exactly why the latest CPU's have features to accelerate IO for hypervisors.

 

[brendan]

I really think its not too difficult to do both, its trivial to have the code check what kernel the system is running.

Ie if your running the latest kernel, then openvz options are disabled etc.

As for maintaining 2 kernels...i highly doubt there would be too much work involved...so i think the poll should have a 3rd option, vanilla kernel for newer kvm and current kernel for openvz and older kvm

 

[virgona]

As I saw there are so many users using openvz, so for the current business status you should keep it and stay on old stable openvz kernel. For future, virtualization looks like main stream, and container will be minority for special field, like web hosts. So you should at least a version which support upstream kvm tech.

There is no way to avoid two branches if you want to keep openvz and updated kvm. It looks openvz focused on rhel kernel version only, but before Proxmox become a big company like RH or commercial company like SWSoft your cannot do in this way, your need more flash points to attract more customers.

My opinion is maintain two version, keep 1.x with openvz and 2.x with lastest kernel and kvm(maybe with lxc), publish 3.x if openvz support new version update and 4.x for kernel later than new openvz kernel version. There always two versions, one for someones like kvm very much and want the newest features, one for someones wants to run openvz and kvm together.The openvz users should know they will lose some of kvm.

You have to do something to keep Proxmox VE alive, save workload is the second for keep alive. Sorry to be strict, I like Proxmox and I want to push it.

 

[Didelo]

I'm not all against two versions, but I can imagine this would increase the load on the development team. However, if they would create a seperate version for OpenVZ, I would regret it if this version was stuck with the 1.3 release features.

I mean, I really do like 1.3 and it runs great on our production environment. However, the only reason why we don't run 1.4 is already because it has no extra OpenVZ features. It would be really cool to be able to store the OpenVZ containers on DRBD or iSCSI storage, but as far as I know, this isn't possible out-of-the-Proxmox-1.4-box. Even though I don't think OpenVZ technologie is preventing the storage of the containers on DRBD or iSCSI, but maybe I'm missing something here.

All I mean to say is: it would be a shame if development on the OpenVZ side of Proxmox would come completely to a halt, because of a seperate version for KVM.

 

[kaptk2]

It would be really cool to be able to store the OpenVZ containers on DRBD or iSCSI storage, but as far as I know, this isn't possible out-of-the-Proxmox-1.4-box. Even though I don't think OpenVZ technologie is preventing the storage of the containers on DRBD or iSCSI, but maybe I'm missing something here.

You are right OpenVZ can't be stored on iSCSI out of the box with Proxmox 1.4. As far as I understand there is an issue with some OpenVZ support tools that do not support the new storage model. I am not totally sure of the issues, maybe a developer could comment more on exactly what is needed. So this is outside of Proxmox and the issue lies more with the OpenVZ project. OpenVZ won't add the features because not enough people have asked for them. So I think we are stuck until the OpenVZ can be changed.

 

[nilski]

Maintaining two versions seems to me would cost too much development time.
Until now I am happy with the ease, functionality, stability and the roadmap set out.
Unless newer kernels have really compelling features I would stick to the roadmap.
Without Proxmox I wouldn't have touched OpenVZ and given up on KVM a long time ago..

Thnx and keep up the good work.
Cheers!

 

[honmanm]

We have been running VMware server for the past two years, generally happy with it except in the area of I/O performance. The UI changes from 1.x to 2.x were not welcome, though.

The reason I'm looking at Proxmox is the underlying storage management - nifty integration with DRBD. OK, that's KVM only right now... BUT the idea of a combined KVM/OpenVZ platform is really appealing especially for our many websites - it's the flexibility and common UI that does the trick.

The problem I've found with disk I/O in VMware Server is probably also true of KVM, i.e. you get twice the file system overhead - and when those are journalling filesystems one application write may turn into 4 physical writes - and it's the disk seeking that kills performance.

So, a vote for continuing with both OpenVZ and KVM - though if OpenVZ doesn't catch up it will eventually have to be left behind. Oh, and OpenVZ on DRBD would be very nice if it can be arranged.

 

[BiagioParuolo]

What are the new benefits of last KVM?
Why is there not OpenVz into last kernels?

Bye

 

[BiagioParuolo]

If there a lot of pro of last KVM versions vs OpenVz then good for KVM, but is there a migration tool from OpenVz to KVM vms?