Installation Advice

[PaulVM]

We are considering a new installation, so hints are welcomed.
We are planning to use a couple of Dell 750s servers
Initial config will be:
128 GB RAM
2 x 4310 CPU
2 x 480 GB SSD
2 x 1.9 TB NVME (U.2) disks
standard 2 x 1 Gbit NIC
Then We add:
2 x SSD Intel D7 P5520 U.2 7680 GB
And more when needed.

Which will be a proper optimization for this configuration to achieve the best performances?
H355 or H755 controller. There is a reason to prefer one over the other?

Thanks, P.

 

[floh8]

If u wanne go with HCI than ceph is always a good decision, but with 2x 1GBit Netadapter - You won't get far with that.

 

[dignus]

Will you require HA? If not… always go for zfs (plus replication if you have the space). If not.. ceoh, but with a network upgrade.

 

[davemcl]

By all means go with ZFS if you have experience with it, youll likely have to flash your controller to IT mode beforehand.
Otherwise the Dell RAID cards in a RAID 1 or 10 with LVM-Thin will run just fine.
Read the storage docs to understand what your config is and isnt capable of.
Proxmox has been rock solid on my Dell R740/640's

 

[alexskysilk]

without use case, all I know is your list of hardware specs. you're offering a possible answer without ever asking a question.

 

[PaulVM]

More details:
- The need is to install 3-4 really heavy VM (windows servers 2019/2022 horribly optimized for I/O, but we can't argue with the supplier ... so we need to compensate the inefficiency with HW).
- No HA neeeded. VM in local storage. Focus in maximize VMs performances
- In the future we have planned add 10 Gbit NICs, but I think it isn't relevant to our needs

In my idea we install PVE on the 2 x 480 GB SSD using ZFS RAID1
Then use the 2 x 1.9 TB NVME (U.2) disks as local storage. ZFS RAID1? mdadm RAID1? other? which FS?
Or the 2 x SSD Intel D7 P5520 U.2 7680 GB if they perform better.
If more details needed ...
Thanks, P.

 

[alexskysilk]

- The need is to install 3-4 really heavy VM (windows servers 2019/2022 horribly optimized for I/O, but we can't argue with the supplier ... so we need to compensate the inefficiency with HW).

How many cores are used maximally at the same time?

- No HA neeeded. VM in local storage. Focus in maximize VMs performances

Why spec two servers then when one should suffice?

In my idea we install PVE on the 2 x 480 GB SSD using ZFS RAID1

fine choice- but you dont really need more then ~32gb

Then use the 2 x 1.9 TB NVME (U.2) disks as local storage. ZFS RAID1? mdadm RAID1? other? which FS?
Or the 2 x SSD Intel D7 P5520 U.2 7680 GB if they perform better.

Again, why the obsession with technology? how much space do your vms need? generally speaking, you want a larger count of smaller disks for best performance, ex: if your VMs need 800GB each, you need 3.2TB usable (for 4 vms.) using that metric I'd use 8x800GB disks in mirrored pairs zpool, or go with bigger drives if you need room to grow. In neither case does it make sense to have two seperate pairs of different size drives.

 

[PaulVM]

How many cores are used maximally at the same time?

8 core for windows server VM.
2 windows server VM for PVE server
+ some core for LXC containers
8+8+4 = about 20 core

Why spec two servers then when one should suffice?

Distribute the 4 heavy VM between the 2 servers.
Have a backup solution for disaster recovery.

fine choice- but you dont really need more then ~32gb

You are probably right, but I like to have spare space (that always will be filled/used in same way ..), and the only alternative option that I have considered is the BOSS card that costs the same and I don't know (never used).
And I prefer to have system in different disks than storage ...

Again, why the obsession with technology? how much space do your vms need? generally speaking, you want a larger count of smaller disks for best performance, ex: if your VMs need 800GB each, you need 3.2TB usable (for 4 vms.) using that metric I'd use 8x800GB disks in mirrored pairs zpool, or go with bigger drives if you need room to grow. In neither case does it make sense to have two seperate pairs of different size drives.

Probably in the middle term we grow to more disks.
In those server we are plannig to store same samba share via LXC CT (some TBs) and a mail server (also via LXC CT; about 1-2 TB).
We also plan to install PBS to both servers to have a crossed backup system for disaster recovery (and an external backup obviously).
My doubt is how to configure those disks for best preformance.
And if the controller model question matter ...
Tanks, P.