Hi @Jan Wedershoven , I seem to recall a few reports similar to yours. For example: https://forum.proxmox.com/threads/proxmox-cluster-of-21-nodes-falls-out-of-sync-when-a-single-node-is-added-or-removed.153149/
I'd recommend pursuing an official...
Ugh, sorry to hear that. When your support contact calls Proxmox "the operating system", you know things are not going well. That said, for people who invested in storage systems that do not officially support PVE, we recommend hiding the PVE...
"ZFS over iSCSI" is a special scheme that involves root SSH access into the storage appliance, requires the storage appliance to run ZFS internally, and directly manipulates said ZFS. And, of course, exporting the resulting ZFS volumes via iSCSI...
One more question: For the LVM storage that you tested against. Did you have the "Allow Snapshots as Volume-Chain" enabled?
It's critically important to understand whether you are testing on a native LVM LV or a QCOW nested within an LV. The...
What caught my attention was the claim of reaching 400K IOPS inside a VM at queue depth 4. In practice, most NAND-based NVMe devices can’t sustain that level of performance. A partial explanation might be that your sequential I/O pattern...
Also, note that 192.90 is not a private IP space
https://datatracker.ietf.org/doc/html/rfc1918#:~:text=3. Private Address Space
Blockbridge: Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
You should place a switch on the back-end network.
Blockbridge: Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
There is no "backup" traffic in basic PVE cluster. It is something you can add with an external node, ie PBS. Do you mean ZFS replication?
If you do not have shared storage, then the VMs cannot failover when node fails. Are you planning to use...
Hi @pelip , welcome to the forum.
To answer your question directly : it is impossible for anyone to say as you provided insufficient data for analyses.
Note that 10G is an overkill for Cluster communication. Are you running Ceph as well...
Hi @Antrill , welcome to the forum.
Can you post your fio job definition?
Blockbridge: Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
No , ZFS is not a Cluster Aware file system. Take a look at storage Wiki page for PVE for shared storage recommendations.
https://pve.proxmox.com/wiki/Storage
Blockbridge: Ultra low latency all-NVME shared storage for Proxmox -...
Great!
Feel free to mark this thread as solved then by editing the first post and selecting the prefix from subject dropdown. That will help to keep forum tidy.
Blockbridge: Ultra low latency all-NVME shared storage for Proxmox -...
There isn’t a single answer that fits all environments, or even a set of best practices you can apply formulaically. The sizing really comes down to:
The number of physical CPUs you have today
The number of virtual cores you’ve provisioned
Your...
Glad I could help. You can now mark this thread as Solved to keep things tidy. You can do this by editing the first post and updating the tag via dropdown near the subject
Blockbridge: Ultra low latency all-NVME shared storage for Proxmox -...
Hi @mjam3204 , welcome to the forum.
An IDE bus is not hot-plug aware. You can try switching to SCSI connected ISO for better luck
Cheers
Blockbridge: Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Hi @kolson3208 , welcome to the forum.
You are likely misremembering. There is no universal "pve" entry in the left tree. You may have had one, ie storage pool name or VE name, but we wouldn't know about it.
Your storage has question marks...
It occurred to me, now that you have vendor confirmation, you can build a test PVE (perhaps even with nested virtualization). Build a VM where you can generate traffic that will overwhelm the backup throughput and temporary location. This should...
I believe for FC boot the advice has always been to install PVE over Debian: https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_13_Trixie
And in this case, you'd use standard Debian instructions for SAN boot.
Blockbridge : Ultra low...
Sounds like my theory was right :-)
That said, I believe PBS , which uses similar approach (or should I say was the first to use this approach) , has this situation under control. IMHO a backup solution that leaves your data at risk when host...