Installing on Dell R710 with 8 500G Drives

[Dayve]

I picked up a Used Dell R710 on the weekend to replace my desktop server running Proxmox.
The dell has 8 SAS 500G drives with a perc 6/i controller. 2 x Intel 5650 6 core processors with 128G of ram.
What is the best way to install Proxmox on this machine?

I was thinking to have two Raid 10's. (4 x 500G drives for OS, CT's and VM's) and (4 x 500G drives for Storage). Should I use the perc 6/i raid controller in the server or should I use Proxmox ZFS as the controller? Or would there be a better way?

 

[Stoiko Ivanov]

HW-RAID controllers are not supported for ZFS (they do lead to quite a few problems in some setups).
Depending on your budget and need for dataintegrity (I would definitely recommend ZFS over HW-RAID for the latter) you can:
* create a HW-RAID and setup PVE using that
* get a HBA (or flash the controller to IT-mode in case that's supported with the perc 6/i) and install ZFS on that

I hope this helps!

 

[Dayve]

So I went ahead and created a HW Raid6 of all 8 drives. Installed PVE and did not enable zfs. I have installed a few CT's and VM's just trying it out.

All the Ct's and Vm's are installed in local-lvm with type LVM-Thin.



Maybe this is normal but my LVM shows as 99% used.



Is this the right way to be setting up PVE?

 

[ness1602]

I would(already have ) install PVE on 2 disks RAID1 ,and use 6 disks for HW RAid5 or 6(whatever your preference).

 

[Dayve]

I would(already have ) install PVE on 2 disks RAID1 ,and use 6 disks for HW RAid5 or 6(whatever your preference).

Think I will go that way. How did you set up the other 6 disks so PVE can access them for data storage?

 

[ness1602]

Make HW raid, and PVE will see it as one drive, partition it as thin LVM

 

[Dayve]

Thank you. That works perfect.

Cheers

 

[ness1602]

Just be sure that the metadata for that array LVM is okay. Proxmox had a bug earlier, which createad too low metadata size.

 

[Dayve]

Just be sure that the metadata for that array LVM is okay. Proxmox had a bug earlier, which createad too low metadata size.

With nothing on that LVM the metadata is at 120MiB. Is that correct?

 

[ness1602]

I think it will be okay.

 

[Dayve]

So I installed a few CT's and a VM in PVE. They are all on the new Storage drive partitioned as a thin LVM.

Problem is that when I go to the "lxc.rootfs.path = /var/lib/lxc/110/rootfs" I do not see the files.

I was reading on the forums that this is what happens if you don't use ZFS.
Is there a way around it?

 

[Stoiko Ivanov]

try running `pct mount 110` beforhand

 

[Dayve]

try running `pct mount 110` beforhand

Perfect. That worked. Will this continue to work after a restart?

Now on to my next thing.
I have been reading over the docs and I don't see how to mount files from one LXC Container to another LXC Container. Is this possible?

 

[Stoiko Ivanov]

Perfect. That worked. Will this continue to work after a restart?

No - this is more meant as a temporary measure if you need to quickly access or save a file from the container, and not
as a general switch to keep the containers contents available on the host (containers should be isolated from the host)

Now on to my next thing.
I have been reading over the docs and I don't see how to mount files from one LXC Container to another LXC Container. Is this possible?

Again not directly - you could setup a dedicated directory which you want to share between the containers and add that to both containers as a bindmount - see https://pve.proxmox.com/pve-docs/chapter-pct.html#pct_container_storage

I hope that helps!

 

[MarcMorgan]

I have just brought a Dell R710 which has a Perc 6/i i believe - 2x 300GB SAS drives 10K

Installed proxmox on the server but cant seem to get the drives to appear in proxmox, im new the the server would so unsure if im missing something in the BIOS etc?

 

[ness1602]

If you installed proxmox on it ,the you should only see lvm-thin pool.

 

[Rithy Toeur]

I would(already have ) install PVE on 2 disks RAID1 ,and use 6 disks for HW RAid5 or 6(whatever your preference).

Hi, could you let me know, did you install PVE on 2 disks RAID1 and RAID 6 as a physical raid or logical?

 

[sahostking]

Hi

Some SAS disks that you buy on Ebay or Amazon come sometimes with firmware of NetApp on them. We had numerous issues with these. Just ensure that is not the case.

This link can help you fix it which I saved a few years back - just requires low level format due to sector size.

https://forum.hddguru.com/viewtopic.php?f=1&t=35283

Forgot to mention that we struggled doing it with the Perc 6/I in the box. We had to swop it with h200 (IT Mode) temporarily so we could format the disks and then do the low level format and then took the h200 card out and replaced it with the Perc 6/I or H700 and then it worked fine.

Not saying the above is your issue I just find a lot of them have netapp firmware on and some have the wrong sector size which causes these type of issues.

 

[ness1602]

Hi, could you let me know, did you install PVE on 2 disks RAID1 and RAID 6 as a physical raid or logical?

Physical RAID, eg raid card on Thinkserver, etc.