Search results

Considered best practice to mirror the OS drives with small drives. But with a 8-bay R620, yeah, you lose 2 drive bays. I previously used ZFS RAID-6 (still losing 2 drives but any two drives can fail before data is lost). I went ahead and mirrored the OS and setup ZFS RAID-50. One last option...

2-node clusters will need a QDevice or any even number of hosts in a cluster. The QDevice will provide a vote to break a tie since each host will vote for itself. Ideally, you want an odd-number of servers for quorum, ie, 3, 5, 7, etc. QDevice can be any device. For example, I use a bare-metal...

Just make sure the solid-state storage is using power-loss protection (PLP) [enterprise] otherwise going get real bad IOPS.

I don't use any production Ceph clusters using solid-state drives. Yup, it's all SAS 15K HDDs. Don't run Windows either, just all Linux VMs. If going to use solid-state drives, you want enterprise sold-state storage with power-loss protection (PLP). With that being said, I use the following VM...

I use this config in production. YMMV. # Configure Dell rNDC X540/I350 Quad NIC card with 10GbE active and 1GbE as backup # VLAN 10 = Management network traffic # VLAN 20 30 40 = VM network traffic # auto lo iface lo inet loopback iface eno1 inet manual...

For new, Arista. For less expensive option, Mikrotik. Pick your speed, medium (wired and/or optic), and number of ports. I use both without issues at 10GbE. They do make higher speed switches.

I used these guides: https://unixcop.com/migrate-virtual-machine-from-vmware-esxi-to-proxmox-ve/ https://knowledgebase.45drives.com/kb/kb450414-migrating-virtual-machine-disks-from-vmware-to-proxmox/ You will need the .vmdk metafile and the flat.vmdk file for 'qemu-img convert' command.

Depends on use case. I don't use HW RAID, so it's either ZFS for stand-alone or Ceph for cluster (3-node minium but I use 5-node). In either case, I use 2 small storage devices to mirror Proxmox via ZFS RAID-1. Then rest of storage for data/VMs.

That's good info to know. I've changed the VM's disk cache to writeback and will monitor performance. I also have standalone ZFS servers and also changed the cache to writeback. Agree. You want enterprise sold-state storage with power-loss protection (PLP).

My optimizations are for a SAS HDD environment. I don't use any SATA/SAS SSDs in the Promox Ceph clusters I manage. I also don't run any Windows VMs. All the VMs are Linux which are a mix of Centos 7 (EOL this year)/Rocky Linux (RL) 8 & 9/Alma Linux (AL) 8 & 9. For the RL/AL VMs, I set the...

I run a 3-node full-mesh broadcast (https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Broadcast_Setup) Ceph cluster without issues. I flashed the PERCs to IT-mode. Corosync, Ceph public & private, Migration network traffic all go through the full-mesh broadcast network. IOPS for...

May want to look at using an IT/HBA-mode controller. I avoid HW RAID controllers when possible.

May want to look at embedded CPU motherboards using an Intel Xeon-D or Atom. Optionally comes with 10GbE (fiber and/or copper) and/or SAS controller. Supermicro and ASRock Rack makes them. I use the Supermicro configurator at wiredzone.com

Not running any H730 in HBA mode yet but running H330 in HBA in production. No issues with SAS drives. Don't use any SSDs though.

No issues with Proxmox on Dell 13th-gen servers. Just need to configure the PERC to HBA mode for use with ZFS/Ceph. Delete any existing virtual disks before switching the PERC to HBA mode otherwise the PERC will NOT see the drives. Needless to say, update all firmware before clean installing...

Don't see why not. Just don't run any VMs on the dedicated Ceph cluster. Never tried it since I do use HCI Ceph cluster without issue.

I got my 3-node Ceph cluster using a full mesh broadcast topology https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Broadcast_Setup using 169.254.x.x/24 (IPv4 link-local addresses) Obviously does not use any switches. Can't expand it either unless one uses a switch. I run both...

Sounds like your issue is an EOL CPU like mine was https://forum.proxmox.com/threads/proxmox-8-ceph-quincy-monitor-no-longer-working-on-amd-opteron-2427.129613 I've since decommissioned the Opteron servers and replaced them with 12th-gen Dells.

Backblaze issues quarterly reports on failure rates on their fleet of HDDs https://www.backblaze.com/blog/backblaze-drive-stats-for-q2-2023/ I prefer Hitachi. If no Hitachi, then WD Purple HDDs.

I used the following steps on Promox 7. Should be same for Proxmox 8. 1. Make sure disk controller is in IT/HBA-mode 2. Put servers in cluster 3. Click "Install Ceph" button on each node 5. Create OSDs, Monitors, MDS as needed 4. $$$