lxc running in a kvm

[RobFantini]

Hello,
I searched and did not see recent posts on this subject.

For each service we use a virtual machine. most of these use very few resources.
I'd like to consolidate these to run 10 or so of these LXC within one KVM.


Questions:
are there any good reasons to do that?

is it possible to run Proxmox with in a KVM to manage the LXC ?
Or should something like turnkey be used?

 

[LnxBil]

Why not run in Proxmox itself?

If you still want a KVM, yes, that's totally possible. I run Proxmox 4 on KVM in a Proxmox 3.4 cluster and have LXC inside. Of course you have to use the GUI in the KVM and it cannot be integrated (4.2 <-> 3.4). If you have the same version "outside" than "inside" you can create a non-HA cluster and have one GUI to manage, but then again, why not run it on Proxmox itself?

 

[RobFantini]

here are the reasons for us to put these small systems to lxc in a kvm. I'm at the end of a long work day so this may not be perfectly written:

1- we use napp-it/omnios zfs storage. pve uses it for iscsi on zfs and nfs . so migration is possible.

2- setting up lxc on lvm at napp-it is too complex or not possible.

3- these systems do not change a lot.

4- we use the napp-it replication extension on a backup storage system. that uses menu based zfs send / receive. depending on the kvm it runs every 5-60 minutes. some systems just once per day.

If the main storage server breaks then there is some manual work to switch over. Along with great documentation and support . The author of napp-it is very helpful. There is a lot of documentation and help available on napp-it menus along with very helpful links. I fell confident that I can write up a small wiki page with instructions on how to do the change over so that anyone with linux / zfs / pve familarity could do the change over.
There will be no cli work needed to change systems.
napp-it is for zfs what freepbx is for asterisk. or proxmox for managing cluster and vm's.

So I have a backup and recovery system for kvm using napp-it . And I want to use it for more virtual machines. We have 20 lxc's . At least half can go to kvm .

When I get caught up on other projects I'll look at setting up shared storage on proxmox. However intend to still use napp-it .

Before proxmox we used debian lenny, vservers and drbd.

then drbd , openvz and proxmox.

with pve4 there were a lot of really good changes. However my drbd setup was scrapped. I've tried glusterfs and ceph. In the fall will try sheepdog. From what I read of sheepdog [ a couple of years ago ] it will be the easiest to setup and maintain.

 

[RobFantini]

for vlans:

at hardware host: add a nic for every vlan to the pve-in-kvm config. set the vlan tag.

inside pve-in-kvm - add one bridge per vlan, keep the vlan . keep vlan tag blank.

 

[RobFantini]

for a couple of weeks we have had 14 LXC running in a KVM.

I have one issue. a daily cronjob has found fsck issues like this:

Jul 23 16:19:45 nodejs-pro4 kernel: [346326.923355] EXT4-fs (dm-7): error count since last fsck: 18
Jul 23 16:19:45 nodejs-pro4 kernel: [346326.923629] EXT4-fs (dm-7): last error at time 1468602060: ext4_put_super:813


/var/log/syslog
Jul 24 16:21:32 services kernel: [432833.468997] EXT4-fs (dm-8): error count since last fsck: 38
Jul 24 16:21:32 services kernel: [432833.469016] EXT4-fs (dm-7): error count since last fsck: 18
Jul 24 16:21:32 services kernel: [432833.469029] EXT4-fs (dm-7): last error at time 1468602060: ext4_put_super:813
Jul 24 16:21:32 services kernel: [432833.469042] EXT4-fs (dm-8): last error at time 1468602062: ext4_put_super:813

there have been 3 of those emails over last week.

we did not get those when the lxc was run on a physical host.

I assume that my issue may be due to hard disk cache setting.

Currently on the pve host , the disk for kvm pve-in-kvm is set to ' Default (no cache) '

the actual storage is at 'zfs over iscsi'

My question - should the cache setting be changed?

Any other suggestions?

 

[LnxBil]

Hmm ... as long as the host (or guest here) does not crash, the cache settings will not matter and they only differ in performance. I cannot say why there are errors. I never experienced them, yet I never used ext4 for lxc before. I always use ZFS, so there is additional filesystem involved.

 

[RobFantini]

LnxBil:

I just want to make sure I fully understand how you set up file system:

for the pve install running in a kvm - during install you used zfs not lvm/ext4 - correct?

 

[LnxBil]

Yes, I did for this particular environment.