Hi everyone.
Iam planing the following proyect of Proxmox 6.X installation. My customer already bought a really old hardware, that I will detail later on. First of all I want to explain the Network side and services running.
Services running on LAN (Its a Health Care Center)
1- PHP and Mysql database Software on LAN running from about 60 PCs ( Web access). No more than 20 simultaneous.
2 - A custom CRM developed for my customer. Also, no more than 20 simult. conections.
2 - Mail Server Zimbra for that 60 users.
3- PBX Elastix for 60 IP Phones Cisco with SCCP
4- FileServer.
5- Pfsense Load balancer, and OVPN server with Virtualized interfaces trough Cisco Trunk LACP.
Now Iam planning to install Promox 6.X latest on 3 same hardware IBM 3850m2
Four slot XEON QuadCore each ( Cant remember the exact model installed).
Dual Power Supplies
64GB RAM (Max 128GB).
4 Gigabit Quad PCIE x 4 LAN + 2 Onboard Ports = Total 18 Eth.
Hard Drives: (Space required is about between 1TB and 1.5TB)
Each server has 4 bay 2.5inch with RAID controller. I thought could be a good idea to make a
RAID 1 with 2x 146GB 15k RPM IBM + 1 Spare (3 Disks) for Proxmox. (I already have bought the disks, so I must use them)
And for Ceph : buy a NVME m.e. PCIE adapter to PCIE x4 in each Server with 1TB PCIe Disk.
I already know that old PCIE 1.0 2.5 Gbps that comes with this old server can give no more than 1GB/s, but is better that plugging a SSD of 1TB to RAID Smart Array. Doesn it???
And If Weneed more disk space on the future we can upgrade by adding a second one PCIE NVMe adapter to each node.
Network (due to I have bought plenty of Quad PCIE Intel Card, and the server is old for 10GE ceph network, I decide to take advantage of using link agregations for speeding up and giving redundancy for the different links)
In each node the 18 ethernet interfaces will be for:
8xEtherchannel LACP reaching a total of 8 Gbps for CEPH NETWORK
4xEtherchannel LACP 4 Gbps for LAN ACCESS Services
2xEtherchannel LACP 2 Gbps for CROSYNC
2xEtherchannel LACP 2 Gbps for TRUNK will be used only for PFSENSE VM
2xFuture Use / Backup
Networking Cisco Chassis
Core Switch
Cisco 4503-E with double power supply , supervisor of 64 Gbps throughout, with line cards reaching
60 1Gbps Cooper Ports
8 Fiber ports 1 Ggbps ports with Cisco GLC Gbis.
Users Switch
Cisco 4506 with dual Poer supply too. Dual supervisors. 96 ports 10/100 Mbps. for PCs and IP Phones (Cisco 7942 already have internal switch for PC port).
The link between Chassis will be trough a FiberChannel 4 Gbps with Gbics on bouth chassis.
Vlans on both Chassis
Lan1, Lan2, Lan3, DMZ, Corosync, Ceph, VOice, Admin, Wifi, Power users, Wan1,Wan2, Wan3,etc.
What do you think about my set up ???
And now the questions:
I almost have the Netwotk part resolve. 8Gbps Etherchannel Ceph network I think will de enough.
I have doubts on Storage for Ceph (I should go for NVme adapters PCIE with 1TB nvme PCIE option, or go to SSD HD 2.5 inch premium server for RAID controller, I know that the old RAID PCIE controller will limit the SSD speed. I already bought an adapter NVMe PCIE with Hp E900 500GB Nvme disk to make some read/write speed test. The only limit I have read besides the limit of 1GByte/s of PCIE 1.0 is that BIOS is too old to boot from it, but is no problem for my case cause I will boot SO Proxmox on RAID 1 2x146GB 15k.
Would be a good idea to have redundancy during mainteniance of any of the 3 node ibm 3850 and add more disk space.
So....
Add 2 only OSD Node to make a 5 CLUSTER NODES . Would be (Already have bought my customer)
Hp Proliant dl385 with dual opteron Dual Core.
32GB RAM
3x146GB 15k on RAID 1 + 1 spare for Proxmox
Same network config as ibm 3850 nodes (18 eth)
1 or 2 NVMe PCIe Adapters with 1TB disk
Will be necesary to separate Journal and Data in different disks (NVMe with adapters) for Ceph in this Max 100 user Network ???
Well Iam very very exited to this new project, I love proxmox, I already installed a cluster 3 years ago 5.X version on a 3 old Dell 2950 and nowadays they continue working excellent. But this time are double quantuty users. so want to make things perfect !!!
Thanks a lot for your prompt reply !!
Iam planing the following proyect of Proxmox 6.X installation. My customer already bought a really old hardware, that I will detail later on. First of all I want to explain the Network side and services running.
Services running on LAN (Its a Health Care Center)
1- PHP and Mysql database Software on LAN running from about 60 PCs ( Web access). No more than 20 simultaneous.
2 - A custom CRM developed for my customer. Also, no more than 20 simult. conections.
2 - Mail Server Zimbra for that 60 users.
3- PBX Elastix for 60 IP Phones Cisco with SCCP
4- FileServer.
5- Pfsense Load balancer, and OVPN server with Virtualized interfaces trough Cisco Trunk LACP.
Now Iam planning to install Promox 6.X latest on 3 same hardware IBM 3850m2
Four slot XEON QuadCore each ( Cant remember the exact model installed).
Dual Power Supplies
64GB RAM (Max 128GB).
4 Gigabit Quad PCIE x 4 LAN + 2 Onboard Ports = Total 18 Eth.
Hard Drives: (Space required is about between 1TB and 1.5TB)
Each server has 4 bay 2.5inch with RAID controller. I thought could be a good idea to make a
RAID 1 with 2x 146GB 15k RPM IBM + 1 Spare (3 Disks) for Proxmox. (I already have bought the disks, so I must use them)
And for Ceph : buy a NVME m.e. PCIE adapter to PCIE x4 in each Server with 1TB PCIe Disk.
I already know that old PCIE 1.0 2.5 Gbps that comes with this old server can give no more than 1GB/s, but is better that plugging a SSD of 1TB to RAID Smart Array. Doesn it???
And If Weneed more disk space on the future we can upgrade by adding a second one PCIE NVMe adapter to each node.
Network (due to I have bought plenty of Quad PCIE Intel Card, and the server is old for 10GE ceph network, I decide to take advantage of using link agregations for speeding up and giving redundancy for the different links)
In each node the 18 ethernet interfaces will be for:
8xEtherchannel LACP reaching a total of 8 Gbps for CEPH NETWORK
4xEtherchannel LACP 4 Gbps for LAN ACCESS Services
2xEtherchannel LACP 2 Gbps for CROSYNC
2xEtherchannel LACP 2 Gbps for TRUNK will be used only for PFSENSE VM
2xFuture Use / Backup
Networking Cisco Chassis
Core Switch
Cisco 4503-E with double power supply , supervisor of 64 Gbps throughout, with line cards reaching
60 1Gbps Cooper Ports
8 Fiber ports 1 Ggbps ports with Cisco GLC Gbis.
Users Switch
Cisco 4506 with dual Poer supply too. Dual supervisors. 96 ports 10/100 Mbps. for PCs and IP Phones (Cisco 7942 already have internal switch for PC port).
The link between Chassis will be trough a FiberChannel 4 Gbps with Gbics on bouth chassis.
Vlans on both Chassis
Lan1, Lan2, Lan3, DMZ, Corosync, Ceph, VOice, Admin, Wifi, Power users, Wan1,Wan2, Wan3,etc.
What do you think about my set up ???
And now the questions:
I almost have the Netwotk part resolve. 8Gbps Etherchannel Ceph network I think will de enough.
I have doubts on Storage for Ceph (I should go for NVme adapters PCIE with 1TB nvme PCIE option, or go to SSD HD 2.5 inch premium server for RAID controller, I know that the old RAID PCIE controller will limit the SSD speed. I already bought an adapter NVMe PCIE with Hp E900 500GB Nvme disk to make some read/write speed test. The only limit I have read besides the limit of 1GByte/s of PCIE 1.0 is that BIOS is too old to boot from it, but is no problem for my case cause I will boot SO Proxmox on RAID 1 2x146GB 15k.
Would be a good idea to have redundancy during mainteniance of any of the 3 node ibm 3850 and add more disk space.
So....
Add 2 only OSD Node to make a 5 CLUSTER NODES . Would be (Already have bought my customer)
Hp Proliant dl385 with dual opteron Dual Core.
32GB RAM
3x146GB 15k on RAID 1 + 1 spare for Proxmox
Same network config as ibm 3850 nodes (18 eth)
1 or 2 NVMe PCIe Adapters with 1TB disk
Will be necesary to separate Journal and Data in different disks (NVMe with adapters) for Ceph in this Max 100 user Network ???
Well Iam very very exited to this new project, I love proxmox, I already installed a cluster 3 years ago 5.X version on a 3 old Dell 2950 and nowadays they continue working excellent. But this time are double quantuty users. so want to make things perfect !!!
Thanks a lot for your prompt reply !!
