Search results

I stood up a 3-node full-mesh broadcast Ceph Squid test cluster using these instructions https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Broadcast_Setup Zero issues. I did use a 169.254.3.0/24 network for Ceph public, private, & Corosync traffic since 169.254.0.0/16 is a IPv4...

ZFS provides bit-rot protection, snapshots, and compression. I use a H200 (flashed to IT-mode) in a Dell R200 as a bare-metal Proxmox Backup Server using ZFS RAID-1. No issues.

Another option is ZFS replication at intervals. It won't be real-time replication like Ceph though. I've ran Ceph using 1GbE full-mesh broadcast network on 14-year old servers and it worked surprisingly well. I don't recommend it for production though.

It is but I do have experience managing proprietary hyper-converged infrastructure (HCI) platforms at work like VMware vSAN and Nutanix AHV. I'm also involved with another project at work in which they are migrating to Hyper-V HCI. Can't beat Proxmox & Ceph as a HCI platform.

Reason I brought up Supermicro at all was that they offer a range of motherboards from mini-ITX to ATX form factors. You can optionally get 10GbE networking (both wired & optic), SAS controller (for more drives), and either an embedded or socketed CPU. I use a Supermicro 5019S-M-G1585L as a...

I run Proxmox on 12th, 13th, & 14th-gen Dells at work. Can't beat used 13th-gen Dells for the value. For example, the R730xd. Can use the rear drives for mirroring Proxmox and front drives for data/VMs. Can replace the PERC with HBA330 for true IT-mode storage controller. Can optionally get...

While it's true PERCs do support HBA passthrough, they use the megaraid driver. Whereas, a Dell HBA, like a HBA330, uses the much simpler mp3sas driver. I swapped out all PERCs from 13th & 14th-gen Dells and replaced them with HBA330s. No issues.

I run a 10GbE Ceph in production. Since Ceph is a scale-out solution, I would make a 7-node Ceph cluster and use the other server as a bare-metal Proxmox Backup Server or use it a replacement server if other nodes fails. Just make sure all hardware is the same (RAM, CPU, NIC, Storage, firmware...

I run a 5-node R720 Ceph cluster in production. I made sure to flash the PERC to IT-mode using this guide https://fohdeesha.com/docs/perc.html Ceph is scale out and loves lots of OSDs. Not hurting for IOPS with workloads ranging from DBs to DHCP servers. I use the following optimizations...

Way better off flashing the H310 to IT-mode and using ZFS RAID. H310 RAID by default has very low queues which leads to terrible IOPS. Flashing it to IT-mode gets you the max queues. Flash using this guide https://fohdeesha.com/docs/perc.html I have a fleet of 12th-gen Dells PERCs flashed to...

Typically with enterprise servers, SAS HDDs have write cache disabled because they are configured to be used HW RAID with a BBU. The HW RAID has built-in RAM cache. I too had terrible IOPS especially with Ceph when used with an IT-mode disk controller, ie, Dell H310 flashed to IT-mode or Dell...

Technically it should work. May want to ask your question at the Promox sub-Reddit forum. I do run production Proxmox Ceph clusters at work on Dells with no issues. I just made sure all hardware (RAM, CPU, NIC, Storage) is the same and running an IT-mode disk controller.

A 3-node Proxmox Ceph PoC does work. Don't recommend it in production in which you want a minimum of 5-nodes, so can lose 2 nodes and still be in business. I do have BOSS-S1 setup in ZFS RAID-1 to boot Proxmox. I use a Dell HBA300 for "true" IT-mode functionality. While 64GB will work for a...

I run H310 flashed to IT-mode in production. Zero issues. And yes, queue depth does matter.

Migrated a production 5-node 16-drive bay R730 VMware cluster over to Proxmox Ceph. Swapped out the PERC H730 for HBA330 for "true" HBA mode. Updated HBA330 to latest firmware. First two drives are Intel DC S3710 SATA boot drives using ZFS RAID-1. Rest of drives are OSDs. Workloads range from...

My reply also works for 8.2.x https://forum.proxmox.com/threads/d...t-works-with-vmware-pvscsi.144806/post-652170

You'll want to see my reply here https://forum.proxmox.com/threads/best-practices-for-setting-up-ceph-in-a-proxmox-environment.148790/post-673329 Since migrating off of VMware to Proxmox, no issues besides the typical drive dying and needing replacing. Will be migrating a 15-host VMware...

I believe 64GBs is the minimum. Don't think 32GB is enough. I do have a server using 100GB SSDs to ZFS RAID-1 Proxmox. So, 128GB is plenty. Since ZFS is a volume manager just like LVM, I would skip the LVM and use ZFS. You can JDOD ZFS, ie, ZFS RAID-0. Of course, backup it up.

Considered best practice to mirror the OS using RAID-1. In my case, I use two small drives to ZFS RAID-1 Proxmox itself. As for the rest of the storage, it depends on the use case. If a single drive, I do use ZFS RAID-0 (stripe), so can use snapshots, rollbacks, compression, error checking...

I found installing Proxmox in UEFI mode on 12th-gen Dells is unpredictable. Luckily, the VMs are already in BIOS mode, so just installed Proxmox using BIOS mode. However on 13th-gen Dells, Proxmox UEFI install with Secure Boot enabled works fine. Go figure.