Choosing the Right Network Infrastructure for a Growing Proxmox Environment

[rosiecharles]

Hi everyone,

I'm planning a Proxmox environment that may need to scale over time, and I'm trying to understand how others approach network infrastructure planning.

Some questions I'm thinking about are:

• Do you deploy 10GbE from the beginning or upgrade later as workloads grow?
• How do you plan to switch capacity for future expansion?
• Do you separate management, storage, and VM traffic using dedicated VLANs?
• What common networking mistakes would you recommend avoiding in a new deployment?

I'd love to hear about your real-world experiences and best practices.

Thanks!

 

[cwt]

Double posting:

R

Thread 'Planning network infrastructure for virtualization and future expansion'

Tuesday at 13:08

Hi everyone,

I'm working on learning more about virtualization and network infrastructure planning.

For those running Proxmox in production, how do you design your network to support future growth? Do you separate storage traffic, management, and VM traffic from the beginning, or keep things simple until expansion is needed?

I'd also be interested in how you choose between 2.5GbE and 10GbE, especially for environments that may grow over time.

I'd love to hear about your real-world experience and any lessons you've learned.

Thanks!

• rosiecharles
• 10gbe firewall linux-bridge networking proxmox virtualization
• Replies: 7
• Forum: Proxmox VE: Networking and Firewall

 

[louie1961]

• Do you deploy 10GbE from the beginning or upgrade later as workloads grow?

For me, being I am in a home lab, I grew my network over time as I learned. Kind of a mistake financially. I would have been better off investing in the 10gbe network up front. But I managed to recoup most of my network investment on resale of my 2.5gbe networking equipment.

• How do you plan to switch capacity for future expansion?

I am not sure what you are asking. Are you asking how I plan for switch capacity? If so, Its a bit of a non issue for me, since I will not likely add many more hosts. A 16 port switch was enough for a while, but I did recently add a cheap 1gbe 8 port switch for my BMC connections and some other non critical networking connections, like my Ring alarm. Otherwise my main switch has 8 copper RJ45 ports that will do 1, 2.5, 5 and 10gbe networking, with auto negotiation and 8 SFP+ ports, that my servers use

• Do you separate management, storage, and VM traffic using dedicated VLANs?

Yes, absolutely. I manage VLANs in my pfSense firewall/router, not in Proxmox. I have a few basic VLANs: trusted (internal services only), untrusted (anything that connects to the internet, like my cloudflare tunnels and wordpress websites) Management, Guest, IoT and Televisions. My storage VLAN exists in L2 only, on the switch. All IP addresses are hard coded. This VLAN doesn't exist on the router, and it is completely handled in the switch (non routed). If I had money to blow and a bigger rack, I might even consider getting a dedicated switch for this and going 25gbe or 40gbe. Not that my workloads need that kind of bandwidth, but sometimes I get upgrade fever.

• What common networking mistakes would you recommend avoiding in a new deployment?

Try to not route as much of your traffic as possible. Inter VLAN routing slows everything down. I also think it is a mistake virtualize your firewall. Its quite possible, and I have done it. But its way more headache than it is worth. Also redundant WAN connections are quite the luxury if you can afford it. For my situation I also don't open any ports on my firewall at all. All web traffic goes out over a Cloudflare tunnel and I lean heavily on the Cloudflare CDN for caching my public web sites, for improved performance. Anything else like Vaultwarden, Nextcloud, Immich, etc. etc., are only externally accesses using Tailscale.