Hi all,
I am looking for some advice on how to best configure my PVE cluster - will really appreciate some guidance!
My current hardware consists of:
Nodes:
- 4 x MS-01 workstations (Intel 13900H, 96GB RAM, 2TB NVMe, 2 x 2.5GBe, 2 x 10GBe)
Switches:
- TP-Link 5 port (10GBe)
- Netgear 5 port (1GBe)
- 2 x Cisco 24 port (1GBe)
NAS:
- Synology RS2423RP+ (1GBe x 2, 10GBe, 12 HDDs - 18TB in total)
Additional hardware:
- 3 x Intel NUC (i5, 8GB RAM) - one is running PBS, with an external SSD connected
- 4 bay HDD enclosure
I am currently storing the volumes on NAS via NFS, although I think that is impacting both performance and network congestion.
I would like to make use of HA / replication, although it sounds like I may need to use CEPH then? Alternativly, if I can get PBS to not be insanely slow with restoration (10+ hours to restore a 1TB windows VM), then restoring from PBS in the event of a failure is also a possibility.
My initial thinking was to try connect the NAS directly to the cluster via the 10GBe ports so that it had direct access to the VM images and would then be both performant and prevent bottlenecking the network, though I was battling to add the NAS directly and ended up connecting it via the router (which obviously kills any 10GBe benefits).
With my current hardware, what would be the most ideal configuration? And should I be looking at storing the VMs via NFS share in the first place, or instead be looking at local storage rather and make more use of PBS after optimising how it's linked?
Current Topology:
Benchmark to PBS:
Any advice would be greatly appreciated!
I am looking for some advice on how to best configure my PVE cluster - will really appreciate some guidance!
My current hardware consists of:
Nodes:
- 4 x MS-01 workstations (Intel 13900H, 96GB RAM, 2TB NVMe, 2 x 2.5GBe, 2 x 10GBe)
Switches:
- TP-Link 5 port (10GBe)
- Netgear 5 port (1GBe)
- 2 x Cisco 24 port (1GBe)
NAS:
- Synology RS2423RP+ (1GBe x 2, 10GBe, 12 HDDs - 18TB in total)
Additional hardware:
- 3 x Intel NUC (i5, 8GB RAM) - one is running PBS, with an external SSD connected
- 4 bay HDD enclosure
I am currently storing the volumes on NAS via NFS, although I think that is impacting both performance and network congestion.
I would like to make use of HA / replication, although it sounds like I may need to use CEPH then? Alternativly, if I can get PBS to not be insanely slow with restoration (10+ hours to restore a 1TB windows VM), then restoring from PBS in the event of a failure is also a possibility.
My initial thinking was to try connect the NAS directly to the cluster via the 10GBe ports so that it had direct access to the VM images and would then be both performant and prevent bottlenecking the network, though I was battling to add the NAS directly and ended up connecting it via the router (which obviously kills any 10GBe benefits).
With my current hardware, what would be the most ideal configuration? And should I be looking at storing the VMs via NFS share in the first place, or instead be looking at local storage rather and make more use of PBS after optimising how it's linked?
Current Topology:
Code:
- 4 MS-01 machines via 2.5GBe to Netgear (2.5GBe) (management VLAN) and via 10GBe to TP-Link (10Gb) (all VLANS via trunk on cisco switch)
- TP-link and Netgear connected for access to routing
- Netgear connected to Cisco Switch -> router
- NAS connected to TP-Link (10GBe)Benchmark to PBS:
Any advice would be greatly appreciated!
Last edited: