Promox and Sandy Bridge Support

[shoey]

I Am looking at setting up a box for testing.

I am looking at getting the Asus P8P67 Motherboard with the i7 2600k and 16GB Ram.

Would this be a good base to start with, is anyone using sandy bridge with proxmox?

 

[MasterTH]

i'm using a i5-2400 very fine processor, but keep in mind, the best performance is nothing without a good raid-controller.


There's one bug, (maybe its mainboad driven, i've the P8H67-M), when Proxmox is started and i disconnect the vga-cable, the kernel disables the irq16 and this is fatal. I think, that the Kernel gets the message, "graphiccard is no longer used, u can disable it" but he not really looks where the graphiccard is. so the kernel disables the pci-x16 slots BAD really bad. after this u have lost 99% of performance (if your raid-controller is on this irq) and u've to reboot the server.


But booting the server without vga-cable connected (which is normal in my optinion) everything work fine, for multiple days (no time to test longer)


bogomips are not as fast as i7-870, but regex/seconds is up to 30% faster

 

[shoey]

Cool i assume it works fine you didn't have to track down any drivers. I also the motherboard you have as it uses the GPU on the motherboard would use some system ram for graphics.

would you have any recommendations for raid cards, as i assume the intel raid on board isn't up to much.

 

[MasterTH]

adaptec is really fast
had some 3ware controllers 9690SA, but they were not as fast as adaptec (under Windows OS)


depending on how much drives u wonna use, starting from 2405 without bbu (recommended to use no write cache)
i prefer 5405/5805 with BBU and write cache enabled, will be a very good combination

 

[shoey]

I was considering putting one disk on the motherboard for Proxmox and then a raid for the VM images. Possibly a raid 10 with 4 drives or a 3 drive raid 5. I not too sure what would give me the best performance. Ijust want a little bit of fault tolerance. As it is only going to be a VM for development use, so maximum uptime isn't so important.

 

[MasterTH]

raid 10, but i'd install the proxmox system on a raid system too

 

[mmenaz]

The problem being that you can't have a separate LVM storage, so all KVM that you create are "file based". Would love if proxmox installer could create the "local" LVM storage not taking the FULL available space (i.e. creating a partition and using it), leaving the remaining space for additional LVM, so you can create KVM VMs with LVM image format, that should perform better than .raw .
I've 2 installation with proxmox + openvz vm on SATA and all LVM storage in RAID (I suggest raid10), and seem to work fine, but heavy I/O involving the sata is really a pain (btw, anyone tried some SSD for this?).

 

[shoey]

I have seen some SSDs used with ESXi and the results a pretty solid. This was using the first generation intel SSD's, i would like to try out some that use the sandforce controller as they have a better I/O throughput.

With the disk images on one disk i wouldn't be so sure if the write usage across all the storage cells on the SSD would be even, and would check to make sure trim is supported. I believe trim support was in the 2.6.33 kernel.

 

[kobuki]

i'm using a i5-2400 very fine processor, but keep in mind, the best performance is nothing without a good raid-controller.


There's one bug, (maybe its mainboad driven, i've the P8H67-M), when Proxmox is started and i disconnect the vga-cable, the kernel disables the irq16 and this is fatal. I think, that the Kernel gets the message, "graphiccard is no longer used, u can disable it" but he not really looks where the graphiccard is. so the kernel disables the pci-x16 slots BAD really bad. after this u have lost 99% of performance (if your raid-controller is on this irq) and u've to reboot the server.


But booting the server without vga-cable connected (which is normal in my optinion) everything work fine, for multiple days (no time to test longer)


bogomips are not as fast as i7-870, but regex/seconds is up to 30% faster

Which kernel are you using? I want to use 2.6.32 for a planned similar setup.

 

[shoey]

Which kernel are you using? I want to use 2.6.32 for a planned similar setup.

I am using Kernal 2.6.32, i havnt raided the machine i havnt had the IRQ issue, as the machine is plugged into a KVM switch, although i will be looking at a clustered setup in the near future.

 

[kobuki]

Just a quick update. I've assembled the updated system, ran a few tests. All is working as it should, except for the coretemp module. I need it at least to confirm the cores are working at a reasonable temperature. Would it be possible to backport the coretemp driver to the 2.6.32 PVE kernel?

Some info:

pveversion -v:

pve-manager: 1.8-15 (pve-manager/1.8/5754)
running kernel: 2.6.32-4-pve
proxmox-ve-2.6.32: 1.8-32
pve-kernel-2.6.32-4-pve: 2.6.32-32
qemu-server: 1.1-30
pve-firmware: 1.0-11
libpve-storage-perl: 1.0-16
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-11
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.14.0-2
ksm-control-daemon: 1.0-5

pveperf:

CPU BOGOMIPS:      54583.50
REGEX/SECOND:      1367769
HD SIZE:           19.69 GB (/dev/mapper/pve-root)
BUFFERED READS:    408.15 MB/sec
AVERAGE SEEK TIME: 6.85 ms
FSYNCS/SECOND:     2892.51
DNS EXT:           75.59 ms
DNS INT:           66.82 ms (...)
(DNS is slow, this is not the production environment.)

aacraid-status:

-- Controller informations --
-- ID | Model | Status
c0 | Adaptec 2405 | Optimal

-- Arrays informations --
-- ID | Type | Size | Status | Task | Progress
c0u0 | RAID10 | 952G | Optimal

-- Disks informations
-- ID | Model | Status
c0u0d0 | WDC WD5003ABYX-0 | Online
c0u0d1 | WDC WD5003ABYX-0 | Online
c0u0d2 | WDC WD5003ABYX-0 | Online
c0u0d3 | WDC WD5003ABYX-0 | Online
(Write cache enabled.)

/proc/cpuinfo first entry (8 total):

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
stepping        : 7
cpu MHz         : 3411.869
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid
bogomips        : 6823.73
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

PS.: moderators, could you please remove my thread, it's really redundant, I've created it in a haste: here

 

[alain]

I notice that you use "i7-2600", and not i7-2600k, as stated in first post. That is a better choice as 2600k does not support vt-d (direct i/o). See :
http://ark.intel.com/Product.aspx?id=52214

and :
http://ark.intel.com/Product.aspx?id=52213


But I wonder if Sandy Bridge is ready for serious use in server with Xeon and Linux (should wait for 2.6.38 kernel ?).

Alain

 

[kobuki]

Well, choice fell on 2600 because it's a little cheaper (not OC'able, and in a server one shouldn't overclock). I don't need VT-d anyway. Effort has been made (in Debian too) to backport changes to the 2.6.32 branch to support SB, and so far everything seems to work. Bleeding-edge kernels are not really good for production systems, and it's unprobable Proxmox will support them for OpenVZ. OTOH stable Debian uses the 2.6.32 kernel too (for another 2 years...) so I see it more or less guaranteed patches will be backported to be able to support new hardware on the long run.