Promox VE 4.0 Network traffic separation

[themilo]

Would like to thank anyone that takes the time to provide assistance and if there is anything I can assist with, just let me know.

Just getting into Proxmox. Starting with v4.0. Using Debian integrated build of Proxmox ve 4.0 from live install DVD.

At this point, I am very confused and have been reading and beating my head against the table / desk for the last 2+ weeks. Several rebuilds in between have occurred! However, I do like the Promox interfaces so far. Just kind of wish there was a more advanced mode for the custom / fine tuned configurations. The interface can be a little restrictive on network configuration (WebUI doesn't seem to like a bond of teams). Also Debian seems to have some issues with Bonds and the Kernel that hasn't or seems to have not been fixed. I have had to modify the networking startup to reload procps before the Bonds were recognized correctly. Before someone asks, the bonds are connecting to separate vmbr's. Each representing their respective network.

I am trying to configure this new install of PVE4 on a 4 Dell node cluster with a preferable manual configuration for Fencing (Not having any luck on this one). Cluster.conf was removed under v4.x. Corosync.conf does not accept previous configurations (At least according to man pages on system/web).

I would also like to restrict internal communications to private networks and eliminate what should be private sites / webui's 8006) from public interfaces.

I have three (3) vlan'd networks and would like to separate all traffic to their respective networks. Public, Storage, and Management. All network connections are connected to two (2) Cisco 2948G-GE-TX switches running CAT OS. Multicast has very limited support and so I have elected to go with unicast. Storage is to a FAS2040A (multi-controller) NAS. DRAC6 is configured on the management network.

Other than through a possible firewall action is there a way to restrict traffic to a specific network? i.e. cfg, conf, ini....

Have not made up my mind on a subscription at this time. I use VMWare at one of my full time jobs, experiment with CENTOS and KVM in one of my companies, however I would like to have a Proxmox type interface to manage my virtual machines without the expense of VMWare. BTW: I am somewhat of a workaholic and love it!

 

[Q-wulf]

i might be understanding the first question regarding "restrictive of network configuration" wrong, but have you looked at openvswitch ?
https://pve.proxmox.com/wiki/Open_vSwitch

 

[themilo]

Q-wulf, I appreciate the response.

I found it impossible to complete the configuration using the standard networking interface. I ended up manually configuring the physical/hardware interfaces to get them set up for my environment. I was creating bonds with bonds/teams under vmbridges. I would then go in and to make temporary changes within the networking gui to see what the system liked or didn't like. I found that the Proxmox software likes commands within the interface files in certain order and does not like comments within the interface files. One would think that if the system/software recognized a commented out line, that the software would leave it alone. Although this is minor, my OCD does twitch once or twice.

I must admit that I haven't read much on the openvswitch. I wasn't sure it would handle the hardware layer of the physical network connections. My needs for switching after the physical layer are very basic.

The questions I have at this time are on the watchdog fencing and the manual configuration of the DRAC's on the Dell's that I have. From my readings on the forums, I have seen several posts on the manual configuration of the DRACs for other versions of Proxmox. I even saw a post from Tom that stated the Fencing section was outdated. I have even seen discussions on whether or not manual fencing should be incorporated into version 4.x with valid posts for and against.

My concern is that I would like to guarantee that network communication meant to be private networks stays on the private network and public communication stays on the public side. IMHO: Fencing should be private/managment, webgui should be private/management, storage only stays on the storage vlans....

I've noticed that my sites are getting a lot more attention from locations that I am not sure they should be receiving attention from. In the tune of 100 ~ 150K attempts within a day. Of those connections, roughly 40 ~ 60K hits are bots/search engines. i.e. Bing, Google, Yahoo....

Other than hooking up or configuring a data analyzer to the networks, how do I go about configuring/physically verifying network traffic? Automatic configurations are nice, but they make me nervous. I start twitching.

I will read up on the Open vSwitch.

Just to recap: Will the Open vSwitch help me guarantee that certain types of data is restricted to specific vlans?

Thank you for the info.

 

[Q-wulf]

[...]

I will read up on the Open vSwitch.

Just to recap: Will the Open vSwitch help me guarantee that certain types of data is restricted to specific vlans?

Thank you for the info.

I posted this in another thread earlier today:
http://forum.proxmox.com/threads/24...nux-bridge-to-openvswitch?p=121537#post121537

openvswitch is an alternative to native bridges, bonds and vlans.

probably wanna read this:
http://openvswitch.org/features/
and this:
http://git.openvswitch.org/cgi-bin/...47ff2bf7c16d1d178d604cba793fdfffd7d71;hb=HEAD

for your questions in the end you can e,g, use netflow or sflow to analyze the traffic going over your openvswitch bond/bridge/ovsinterface(vlan)

From the the limited benchmarks i have seen openvswitch is nowadays on at least on teh same level if not faster then native linux switch as far as resource utilisation is concerned.



to answer your questions in more detail:
install openvswitch-switch package for proxmox, set up ovsbond, ovsbridge and ovsIntPorts (vlans) via the proxmox gui. Advanced stuff you will most likely need the "nano" on /etc/network/interfaces.

Once that is done, your ovsswitch works just like a regular hardware switch. So if you configure e.g. cluster communication to run on Vlan 10 with ip 10.10.1.1 255.255.255.0 then only clients on that Vlan can get access.

If your a bit on the cautious side, you can create multiple ovsBonds on Interfaces connected to physically different Hardware switches, assign different ovsbridges to those bonds, and different ovsIntPorts to those.
evoila, physical and virtual separation.



ps.: Something that popped while reading your last reply:
Did you read this ? https://pve.proxmox.com/wiki/Network_Model
specifically this part:

The network configuration is usually changed using the web interface. Changes are stored to/etc/network/interfaces.new, and are activated when you reboot the host. Actual configuration resides in/etc/network/interfaces

It probably accounts for your manual mix and match. in the earlier part of said reply.

 

[themilo]

Q-wulf,

Thank you for the reply.

I definitely need to read up on the Open vSwitch. I've spent the last hour assisting customers and have not had time.

If I am able to determine that fencing traffic is not traveling on the management network, where would I go to set that?

and

Under PVE4, how can I set up manual fencing to the DRACs now that the cluster.conf has been removed?

or

Would it be better to run PVE3.x to manage the DRACs? This way I can guarantee that traffic will always be kept on the management vlan.

I just got caught up on a couple articles that you have been posting to: SSD / SSHD.

Initially the conf.new caught me off guard but I was able to figure that out quickly.

To see a build similar to what I have built, but with 4 node, see: https://alteeve.ca/w/AN!Cluster_Tutorial_2#A_Map.21

Again, thank you.

 

[themilo]

Q-wulf,

Thank you for the reply.

I definitely need to read up on the Open vSwitch. I've spent the last hour assisting customers and have not had time.

If I am able to determine that fencing traffic is not traveling on the management network, where would I go to set that?

and

Under PVE4, how can I set up manual fencing to the DRACs now that the cluster.conf has been removed?

or

Would it be better to run PVE3.x to manage the DRACs? This way I can guarantee that traffic will always be kept on the management vlan.

I just got caught up on a couple articles that you have been posting to: SSD / SSHD.

Initially the conf.new caught me off guard but I was able to figure that out quickly.

To see a build similar to what I have built, but with 4 node, see: https://alteeve.ca/w/AN!Cluster_Tutorial_2#A_Map.21

Again, thank you.

 

[themilo]

Q-wulf,

I've gone back in and read up on open vswitch. Thank you for the suggestion.

I'm just not sure if openvswitch will assist me in resolving the questions I have. I do see where is could be very useful and I will most likely try to incorporate it into my VM builds / structure to help eliminate traffic travelling outside of the VM cluster servers.

1. I'm trying to figure out external / manual fencing for PVE 4.0. I would like to be able to use the DRAC's on the system to perform fencing actions. If Proxmox VE 4.0 does not support manual fencing and manual fencing cannot be added, then how can I direct which network port/vlan is used by watchdog fencing?

2. How can I limit or configure the proxy not to respond to web requests on port :8006 on all bridged IP's on a system?

Does anyone have an answer to my questions or am I just being too cautious on my builds / configurations or am I asking too much of this version?

BTW: Was there anything significantly wrong with Proxmox v3.4 as far as security / operations or was version 4.0 a different direction?

v/r,
Milo