Been migrating Dell servers at work to Proxmox from VMware.
While it's true the PERC RAID controller can be switched to HBA-mode, it uses the megaraid_sas driver which has cause issues in the past.
I decided to replace the PERC with a Dell...
In production at work, I use the following:
Intel X550
Intel X540
Intel i350
without issues with latest firmware.
At home, I use a Mellanox ConnectX-3 SFP+ 10GbE fiber NIC without issues also with latest firmware.
I run ZFS on standalone servers and Ceph on clustered servers.
Usually, on Dell servers, the write cache on hard drives is disabled because it is assumed they will be used on a BBU RAID controller.
Since, ZFS & Ceph don't play nice with RAID...
Nope.
All network is traffic going over single network cable per node as physically described (1 -> 2, 3; 2 -> 1, 3; 3 -> 1,2) at https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Introduction
So, yeah, you can say that is a...
I run a 3-node Proxmox Ceph cluster using a full-mesh broadcast network per https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Broadcast_Setup
Each node is directly connected to each other without a switch. Yes, since it's broadcast...
It was quite reproduceable in my production environment with Ceph migrations done every other week. Again, YMMV. All VMs are Linux with qemu-guest-agent installed.
Was only through trial-and-error, I found the cache policies that work for ZFS &...
Through trial-and-error, I use the following disk cache policies in production:
For stand-alone servers using ZFS, writeback.
For Ceph servers, none.
Using anything else other than none/no cache with Ceph cause VM migration corruption issues.
Better off flashing it to IT-Mode per https://fohdeesha.com/docs/perc.html
Do NOT skip any steps and take your time. Make sure you record the SAS address.
Don't forget to flash a BIOS and/or ROM on it if you want to boot from the drives.
Been...
Trying my luck in asking my question in this forum.
Have 3-nodes that are physically cabled (each node has 2 cables to each other node, total 6 six cables, no switch) as a full mesh network per...
The 3-nodes are physically cabled (each node has 2 cables to each other node, total 6 six cables, no switch) as a full mesh network per https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Introduction (4. Broadcast: Each packet is sent...