Need some advice for my little proxmox project please :)

[hugosxm]

Hi everybody, i've been playing with proxmox for some months and really appreciate it ! so i decided to go with this solution for my home server

Now i have to idea and i'm trying to find the best, i'm sure you can help

The server will be used for many vm like firewall, proxy, plex, owncloud, ... and as a NAS ! ( openmediavault )
I have 6 HDD, no raid controller and no ZFS because i don't have ECC ram yet...So the 6HDD are connected directly to my motherboard...

i have found to solution for my need, please help me to choose one....

First idea :

i have just managed to do that today : i installed proxmox on two good usb stick in RAID1, on this ARRAY i have proxmox running and a small raw file for the openmediavault vm. So proxmox and OMV are running from the USB key. I passed the 6HDD directly to OMV because i have Vt-D capable hardware. From the OMV VM ,I created a RAID array that i want to use as a storage for my raw or qcow file from proxmox. So i set up a NFS share on OMV and added it to proxmox. When Proxmox boot, OMV start too and proxmox can find his NFS storage in a minute.

I have made some bench with "dd" and i found that i reach 370Mo/s in write directly on the OMV vm that have the HDD "passed trought" and i lose 70mo/s when I do the same command on the proxmox that use the NFS share. I have already set NFS correctly ( mouting option, MTU, ... ).

Question :

A HDD "passed trought" have the same performance that i would have directly on a physical machine or we loose some IO ?
Why can i reach 300Mo by a single virtual e1000 network card ?!

Second idea ( more simple )

Juste make a raid 10 array with my 6 drive, put proxmox on it and add a lvm as a storage.

Question :

The performance will be FAR better or just a little ?
Is it a problem to have verry large raw or qcow file ? As i said i want to set an openmediavault VM and this VM will have 2.5To of disk size. That's why i wanted to pass the HDD directly to it. I don't know if 2.5To qcow2 or raw file is a good idea...



Please help me with this ^^


Sorry for my english, i do my best, it's easy for me to read but it's more complicated to write...

 

[acidrop]

hi

first , dont install proxmox on usb sticks (its write intensive).

second, whats the benefit of 1st scenario? why omv vm and proxmox on same machine
I think it complicates things with no real benefits.If you need nfs use another box or install nfs server directly on proxmox.

I would prefer 2nd scenario wich is the recommended.LVM has very little overhead over block devices.If you will use linux raid instead of hardware raid that will add another layer of course.

I would not use such big sizes although I think qcow2 supports it.
Prefer raw format or lvm instead of qcow2 if speed is a concern(you will loose live snapshots).
ZFS (zol) is also a very good option (very fast snapshots).ECC is recommended if your vms are for production use though.

regards

 

[hugosxm]

Thanks for you answer !

I will forgot the complicated solution...

FInally i am thinking at this :

I will do a RAID1 /boot of 200M and the other part of the 6HDD will be a RAID10 for proxmox and wm disk...I will choose raw file format.

My RAM is not ECC and i don't have so much RAM, ZFS being hungry of RAM and friendly with ECC i will be patient and wait for some money to buy some ECC Stick and use ZFS

Now there is a last question :

A really want a openmediavault and i only have this machine ! I will buy two HDD to put proxmox and the VM on it and passtrought the 6 other to omv but for now i can't do this. so, is it ok to use LARGE raw file ? Or maybe i can create 4*500g raw file ??

thnaks for your help !!

 

[acidrop]

Now there is a last question :

A really want a openmediavault and i only have this machine ! I will buy two HDD to put proxmox and the VM on it and passtrought the 6 other to omv but for now i can't do this. so, is it ok to use LARGE raw file ? Or maybe i can create 4*500g raw file ??

thnaks for your help !!

If your question is, if you can create a omv vm with 4 .raw files as hard disk for omv to create a RAID config through omv, then yes you can. But that is not recommened. What I really don't understand is why want omv so badly virtualized?

 

[gkovacs]

KVM I/O is single-threaded, also NFS out of the box is not configured to provide good performance, so I would not give VM storage to Proxmox that way.

I think you best bet (performance-wise) without using a HW RAID controller is to install Debian Wheezy first, creating a 6-disk md raid in the installer, and a Proxmox compatible LVM setup on top of that.
After that you can install Proxmox via packages, you should have decent performance.

https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Wheezy

 

[hugosxm]

thanks to everybody !

i fiannly created a 6 HDD raid 10 with lvm on it to store my wmdisk and my proxmox act as my NAS as well I am setting up NFS and SMB on it

Now there is another question :

I need a firewall solution, i tried the proxmox one which i found really cool for VM but i need a real firewall solution because this firewall will be in charge of all my network, not only the virtual machines...

I want to givea try to shorewall, but i don't know WHERE to install it...

Let me explain

I have 3 physical NIC. I want to set a router/firewall with two of thew, one will be connected to my physical lan and all my VM and the other will be the WAN. So shorewall can manage all the traffic of the physical and virtual network.

i don't know if i have to set a firewall VM or if i have to set shorewall DIRECTLY on my proxmox host, what are the pro and con's ?

i need a proxy too, like squid, where can i set it ?

i am a little bit loss... i really don't know where to set up this.

Have a nice day

 

[acidrop]

hi

It is not recommended having your internet firewall on a vm neither proxmox host.
I would prefer a separate physical device to handle internet like pfsense on a alix or apu board for example.
But if you want to experiment you could try pfsense as a vm with 2 vm nics bridged on the 2 physical nics.

 

[hugosxm]

i can not get another physical device for this purpose at this moment, i downloaded a pfsense iso yesterday to try it out...

you said : "It is not recommended having your lans network on a vm neither proxmox host."

what is the best option of this two ??

 

[acidrop]

sorry I corrected that.I meant your lans internet firewall.
Try pfense as a vm as I told you.It should work as you expect.

 

[gkovacs]

i can not get another physical device for this purpose at this moment, i downloaded a pfsense iso yesterday to try it out...

you said : "It is not recommended having your lans network on a vm neither proxmox host."

what is the best option of this two ??

Actually, having a firewall running in a VM is a fairly secure way of doing it, if you can't use an external appliance. PfSense is absolutely fine, but if you need decent performance, you should use the 2.2 version (which is currently in beta), because that's the first release with built-in virtio lan drivers.

You can also use the built-in Proxmox firewall:
https://pve.proxmox.com/wiki/Proxmox_VE_Firewall

 

[hugosxm]

thanks for you help guys !

according to you Setting up a VM machine as a firewall is OK, i will give a try to pfsense AND shorewall this week i will choose virtio NIC to test

have a nice day !

 

[acidrop]

There is virtio support for pfsense 2.1 also but with some limitations like traffic shaping does not work.
https://doc.pfsense.org/index.php/VirtIO_Driver_Support
I'm using e1000 vnics for that reason until pf 2.2 will be available.