Networking Proxmox PVE 4.2 (latest version)

[Gopal]

Hello folks:
first, I must confess that I am new to proxmox. I am also a beginner in networking. I am working hard to learn it. Having said that, I am getting confused by all the stuff that I am reading in this forum and the wiki. I think my needs are simple, but I don't have a clue how to proceed.

I am hoping that someone can give me simple answers ...
goal: I am building a lab ...

Here's my setup ..

1) Server: Dell Poweredge T605
2) 2 quad core cpu's
3) 32gb RAM
4) 2 HD's 500GB each
5) NIC cards:
a) onboard - 1 port
b) NIC card - 2 ports
c) NIC card - 1 port

So, in total there are 3 physical NIc cards. One of them is a dual port.

Cabling:

1) attached a cable from one of the dual port NIC to my laptop
2) attached a cable from my ISP cable modem to the nic on board. I have been playing around to see which one I should use. I don't have a clue

Installed ProxMox VE (4.2, I believe) whatever the latest download says.

After installation, I got the management IP as 192.168.2.100100.2:8006. This allows me to go to the management of the VM's.

I installed about 4 vm's. (2 ubuntu, 1 kali, 1 windows 7)
I did not change any default configuration. Its all confusing, I don't understand.

When I log in to ProxMox on a console, I tried to ping 8.8.8. and I am not able to ping out side.

So, this is causing a problem because I am not able to do apt-get install openvswitch ...

How can I solve this problem? I am confused as heck.

I would prefer to have the following:

Outside --> SOPHOS FIREWALL UTM (VM on proxmox server) --> VYOS router (VM on proxmox server) -->
3 vlans

The SOPHOS UTM is looking for 2 interfaces. Also, I don't have openvswitch and I am trying to get that installed. I am stuck because I cant get access to the internet.

vlan1 - lab1 with internal ip 192.168.10.?
vlan 2 - lab2 with internal ip[ 192.168.20.?
vlan3 - lab3 with internal ip 192.168.30.?

I would like to put a couple of vm's in each.

I know that what I have listed above is easy to some of you who understand networking and proxmox well or even understand virtualized environment. However, I am having a hard time even imagining this setup.

Will someone help me by giving me some idea on how I can do this in a way that I can understand, please?

I apologize in advance if this scenario has been addressed in the forum. I appreciate your help and guidance.

Thanks in advance,
Gopal

 

[Gopal]

I apologize ...
the management ip is 192.168.100.2:8006. it is the default that ProxMox gives during installation.

My ISP (comcast) gives me an external IP as 10.0.0.? for my wireless connection ...

 

[Gopal]

???

 

[alexskysilk]

lets take a big step back.

From what you describe, your 4 network ports are not being utilized for any practical purpose. unless you intend to have them act like switch ports, dont even configure any except 1, which is the one connected to your router/switch.

your laptop should be connected to your router/switch as well.

all your vlans should be connected to your externally connected port, although its probably not necessary to do any configuration since you can just specify the vlan on the VM virtual NIC connection.

Configuration for virtual UTM:

this is possible. the configuration should be as follows:

create two bridges, we'll call them vmbr0 and vmbr1. connect vmbr0 to your physical nic attached to your router, and leave vmbr1 without any NICs connected. attach your UTM to both bridges. Connect all your VMs to vmbr1. If the intention is for the UTM to protect your physical network as well, then vmbr1 should be attached to another ethernet connection which, in turn, is to be connected to your switch (NOT THE ROUTER.)

The network diagram for this config would look like this:

[router] -- [eth0] -- [Dell Poweredge] -- [eth1] -- [network switch]
10.0.0.1 (vmbr0) 192.168.100.2 (vmbr1)

UTM NIC1 attached to vmbr0, DHCP
UTM NIC1 attached to vmbr1, IP 192.168.100.1 (DHCP server)

 

[Gopal]

lets take a big step back.

From what you describe, your 4 network ports are not being utilized for any practical purpose. unless you intend to have them act like switch ports, dont even configure any except 1, which is the one connected to your router/switch.

your laptop should be connected to your router/switch as well.

all your vlans should be connected to your externally connected port, although its probably not necessary to do any configuration since you can just specify the vlan on the VM virtual NIC connection.

Configuration for virtual UTM:

this is possible. the configuration should be as follows:

create two bridges, we'll call them vmbr0 and vmbr1. connect vmbr0 to your physical nic attached to your router, and leave vmbr1 without any NICs connected. attach your UTM to both bridges. Connect all your VMs to vmbr1. If the intention is for the UTM to protect your physical network as well, then vmbr1 should be attached to another ethernet connection which, in turn, is to be connected to your switch (NOT THE ROUTER.)

The network diagram for this config would look like this:

[router] -- [eth0] -- [Dell Poweredge] -- [eth1] -- [network switch]
10.0.0.1 (vmbr0) 192.168.100.2 (vmbr1)

UTM NIC1 attached to vmbr0, DHCP
UTM NIC1 attached to vmbr1, IP 192.168.100.1 (DHCP server)

 

[Gopal]

Thanks for your reply.

I think I understand what you are saying. Yes, I want SOPHOS/UTM to protect my internal network. When you say "switch", are you referring to VYOS?, in which case, I would need to install vyos first ...

I am confused by "switch". Are you referring to proxmox ve interface (vmbr1, vmbr2 ...)? Actually, I would prefer VYOS to do all the routing etc ...

I connected both eth0 physical nic to my cable modem and eth1 physical nic to my cable modem. Probably, I must not connect any more physical connection to the cable/modem ....
I did create 2 bridges like you said

having said that,

vmbr0 -- > eth0 - 10.0.0.xx
vmbr1 --> eth1 - 10.0.0.xx (sophos/utm). More than likely, this is incorrect.

As per your suggestion, I will disconnect eth1 from the cable modem/router and remove the IP from vmbr1. so, new configuration will look like this

internet - [cable modem/router] -- [eth0/vmbr0 - 10.0.0.xx] -- [Dell Poweredge - physical NIC] --> eth1/vmbr1 (no ip)

Anyway, I am doing it wrong. Not like what you have said. I think I have it backwards. The best part of this learning is that I am able to tear down everything and do it from the beginning.

This is an interesting learning experience for me. I am trying hard to imagine a real CISCO ASA 5505 firewall router (outside interface, inside interface and a dmz) etc . and mapping it in mind to proxmox ve interfaces. I noticed that there are many different ways to create this network and I am getting overwhelmed and confused.

So, in short,

my eth0 - outside interface - vmbr0 (this is the default). This interface connects to the internet. It would be nice to install SOPHOS UTM here and connect to the inside interface as well. I would like the UTM to connect to the VYOS router to route traffic. My lan will be behind the VYOS router. VYOS router will create my vlans and deliver ip address to its clients.

I am confused by where to install these vm's for management ...
1) sophos management - perhaps eth0/vmbr0
2) vyos management - not sure
3) proxmox ve management - not sure (but definitely the inside network mayb e 192.168.10.xx). right now its on eth0, available to outside.

The above 3 confuses me. right now, I have the proxmox ve management on my external interface. What this means is that it is available to anyone. security risk???

I also noticed that if I change the ip address of eth1/vmbr1 from 10.0.0.xx to 192.168.10.2, the traffic is not getting routed to the outside interface (eth0/vmbr0) and I am not able to use the internet. So, I need to learn the proxmox nat configuration from the wiki.

I don't want proxmox doing routing etc...I want to use proxmox to just manage my vm's.
I would like SOPHOS/UTM to provide security and VYOS to provide all the routing ...



I don't know if it makes any sense ...

I installed proxmox ve management to the external interface (eth0) because I thought I wanted to manage it from my laptop using my wireless connection to the cable modem. the server is downstairs and I wanted to manage it from upstairs. That's all.

I don't know if there is a best practice for configuration a virtual environment.

Am I right in wanting to present the sophos utm firewall/router to the outside interface (eth0/vmbr0)? If so, I would like to route all traffic thru the UTM to and from the internet. I would like to make sure that the only way to get to the sophos/utm is by ssh, or vpn connection. This way I can connect to my sophos utm and my get to my proxmox ve management and all of VM's from the outside world! That's my goal.

I am not sure how to do that? I am just confused and I am sure it is basic and simple. I apologize for the long reply and maybe, repeating stuff ...
I am still learning.
Thank you for your time and help and I appreciate you sending me a reply.

Gopal

 

[alexskysilk]

When you say "switch", are you referring to VYOS?

No, I mean a network switch.

I am confused by "switch". Are you referring to proxmox ve interface (vmbr1, vmbr2 ...)? Actually, I would prefer VYOS to do all the routing etc ...

I'm not sure why you're referencing vyos. The Sophos UTM will perform routing duties for you; why do you think you need it and a vyos appliance?

vmbr0 -- > eth0 - 10.0.0.xx
vmbr1 --> eth1 - 10.0.0.xx (sophos/utm). More than likely, this is incorrect.

the subnets need to be different on each bridge.

I am confused by where to install these vm's for management ...
1) sophos management - perhaps eth0/vmbr0
2) vyos management - not sure
3) proxmox ve management - not sure (but definitely the inside network mayb e 192.168.10.xx). right now its on eth0, available to outside.

remember that in you're operating in a virtualized environment. As such, you have two virtual "switches" (bridges), vmbr0 and vmbr1. vmbr0 is only connected to eth0 and a vnic on your Sopohs vm. vmbr1 is connected to everything else including another vnic on your sophos VM (yes, this VM needs to have two nics), with 192.168.10.xx subnet. If you want proxmox to be manageable by your internal network, give vmbr1 an address (which will be your proxmox management address.)

 

[Gopal]

No, I mean a network switch.


I'm not sure why you're referencing vyos. The Sophos UTM will perform routing duties for you; why do you think you need it and a vyos appliance?


the subnets need to be different on each bridge.



remember that in you're operating in a virtualized environment. As such, you have two virtual "switches" (bridges), vmbr0 and vmbr1. vmbr0 is only connected to eth0 and a vnic on your Sopohs vm. vmbr1 is connected to everything else including another vnic on your sophos VM (yes, this VM needs to have two nics), with 192.168.10.xx subnet. If you want proxmox to be manageable by your internal network, give vmbr1 an address (which will be your proxmox management address.)

Thanks again for your reply. I like the way you are explaining stuff to me. Its slowly starting to sink in. The paradigm shift is difficult for me. I am having to imagine a physical device all the time and then, translate it to virtual environment. But, I am learning. I appreciate your time and helping me with this setup.

VYOS - I don't really need it. You are right, I don't really need it ...
I thought I needed it to create vlans. In my class, we create a lab using vyos and create vlans. That's why I thought I needed it. It doesn't matter.

Currently, I have the following on ProxMox VE (yes, I keep changing as I learn more. I keep rebuilding it). I think I have it correct this time.
vmbr0 - eth0 - 10.0.0.xx with 10.0.0.1 as the gateway - outside intercace via cable modem ... not touching this
vmbr1 - 192.168.10.2 - internal network 1 - management network
vmbr2 - 192.168.20.2 - internal network 2 - lab portal - will host about 10/12 vm's - pen testing lab
vmbr3 - 192.168.30.2 - internal network 3 - general/learning portal will host a couple of vm's

All 3 networks will need to connect to the internet ...

currently, the proxmox mgmt is on vmbr0/eth0 with ip 10.0.0.xx
I will eventually change that to vmbr1 (which I am calling the management network)...

I am working on installing SOPHOS UTM. During installation the setup asks me where the mgmt is located. Choice is eth0 or eth1. I think I was getting confused with this question. I am getting confused with eth0 (vmbr0 bridge in proxmox) to eth0 (vnic in sophos utm) with ip 192.168.100.2 which is the default. It does show 2 nics (eth0 and eth1) during setup. So that part is good.

SOPHOS UTM eth0 - 192.168.100.2 - mgmt --> this is the part which is confusing. eth0 on proxmox has ip 10.0.0.xx and eth0 on SOPHOS has ip 192.168.100.2 (if I leave it at the default). If I were to change the IP address from default to 10.0.0.xx in the same network as proxmox vmbr0, it will be available to me to configure on my wireless network and the internet.

I have not even begun using tghe web interface to start my configuring of sophos .... I am still in the installation phase.

I am guessing the following

physical machine #1 - Proxmox VE (Dell Poweredge) vmbr0/eth0 - 10.0.0.x, vmbr1/eth1 - 192.168.10.2, vmbr2,vmbr3 ...
physical machine #2 - SOPHOS UTM eth0 - connected to eth0 of proxmox machine, eth1 connected to eth1 of proxmox ...

In the above case, eth0 of sophos utm will have an ip address in the same network of proxmox eth0 which is 10.0.0.xx ... Is my imagination correct?

Logically, this is not making any sense to me. 2 interfaces with same IP. I think I am missing something in your explanation. Could you please comment on the above paragraph please?

SOPHOS UTM eth1 - will have vmbr1 ip - 192.168.10.3 or something similar within the network 192.168.10.0

I have cancelled the setup and will rerun it again after making sure that my ips are ok in proxmox ...

Thanks for your time and patience with me.

 

[alexskysilk]

physical machine #1 - Proxmox VE (Dell Poweredge) vmbr0/eth0 - 10.0.0.x, vmbr1/eth1 - 192.168.10.2, vmbr2,vmbr3 ...
physical machine #2 - SOPHOS UTM eth0 - connected to eth0 of proxmox machine, eth1 connected to eth1 of proxmox ...

You only have one physical machine, and that is your dell poweredge. You have two physical network connections on your physical machine, eth0 connected to your modem and eth1 connected to a physical switch (NOT THE MODEM, lets call this pswitch0) this represents the totality of your physical connections.

LOGICALLY, you have two switches (or 5, more on that in a bit.) switch 1 (vmbr0) is connected to your physical eth0, and vmbr1 is connected your physical eth1.

In the above case, eth0 of sophos utm will have an ip address in the same network of proxmox eth0 which is 10.0.0.xx ... Is my imagination correct?

correct.

Logically, this is not making any sense to me. 2 interfaces with same IP.

not sure what you mean by that. NONE of your interfaces, physical or logical, will have the same IP. lets run it down:

proxmox eth0: no IP
proxmox eth1: no IP
vmbr0: no IP (unless you want management on the WAN side
vmbr1: 192.168.10.2, gateway 192.168.10.1
UTM eth0 (WAN): 10.0.0.2, gateway 10.0.0.1
UTM eth1 (LAN): 192.168.10.1

Now lets talk about your VLANs. I dont know why you want them, but you need to repeat the logical configuration for each vlan (it will need a bridge, attached to the UTM which will have to define vlan interfaces for each. Its possible to do this with just one bridge as vlans managed outside by proxmox.) Also, if you want to be able to use your vlans externally to proxmox you'll need a layer 2 managed switch. If you dont have one or dont know what that means, I suggest you probably dont want VLANs at all.

 

[Gopal]

Hi Alex,
Thanks for your quick reply.

You only have one physical machine, and that is your dell poweredge. You have two physical network connections on your physical machine, eth0 connected to your modem and eth1 connected to a physical switch (NOT THE MODEM, lets call this pswitch0) this represents the totality of your physical connections.

The above was just an example so that I can imagine all the virtual stuff in terms of physical machines. I need to think in terms of actual machines to be able to visualize the virtual environment. So, I was just making stuff in my head and mapping it to the virtual environment based on your explanation. I apologize for confusing the issue.

The dell poweredge has 4 physical nics. 1 on the motherbopard, 1 dual port interface and 1 addition nic.
however, I have connected only 1 of the port from the dual port nic to the cable modem. This is my eth0. I do not have a physical switch at all. So, no pswitch0. Do I need it a physical switch?

LOGICALLY, you have two switches (or 5, more on that in a bit.) switch 1 (vmbr0) is connected to your physical eth0, and vmbr1 is connected your physical eth1.

As I mentioned there is no physical connection to a switch pswitch0. I can get one if there is a need. Actually, I do have a 6 port CISCO ASA 550 with 3 interfaces. Outside (going to cable modem), (used about 3 ports) for inside (which is the 192.168.xx.xx interfaces) and a (about 2 ports) DMZ. I remember using just the default configuration of CISCO. Do I need to use this?

correct.


not sure what you mean by that. NONE of your interfaces, physical or logical, will have the same IP. lets run it down:

Oh, I was referring to the fact that if I had left the default configuration for eth0 during setup it will have the ip for eth1 on eth0. Actually, don't worry about it. I have confused myself again. The configuration that you have given below is perfect and this is what I have with a couple of exceptions that fall within the logical explanation.

proxmox eth0: no IP
proxmox eth1: no IP
vmbr0: no IP (unless you want management on the WAN side
vmbr1: 192.168.10.2, gateway 192.168.10.1
UTM eth0 (WAN): 10.0.0.2, gateway 10.0.0.1
UTM eth1 (LAN): 192.168.10.1

Awesome. Finally, I have the above configuration. I do have an IP on vmbr0 as the 10.0.0.xx to manage the proxmox VE from my wireless connection on my laptop. The sophos eth0 is on the same subnet with eth0 ip as you have mentioned. The sophos eth1 has the same subnet as vmbr1.

Now lets talk about your VLANs. I dont know why you want them, but you need to repeat the logical configuration for each vlan (it will need a bridge, attached to the UTM which will have to define vlan interfaces for each. Its possible to do this with just one bridge as vlans managed outside by proxmox.) Also, if you want to be able to use your vlans externally to proxmox you'll need a layer 2 managed switch. If you dont have one or dont know what that means, I suggest you probably dont want VLANs at all.

Now, the question I have is for pswitch0 for eth1. Do I need a physical switch for eth1?

Thank you and I appreciate all your help and time.

Gopal

 

[alexskysilk]

Hi Alex,
Thanks for your quick reply.



Now, the question I have is for pswitch0 for eth1. Do I need a physical switch for eth1?

Thank you and I appreciate all your help and time.

Gopal

I cant answer that for you are you intending to connect any other devices to your internal (LAN) network? if so, then yes, you'll need some device to allow you to attach them to your LAN. If you dont, you dont even need to configure eth1 at all.

 

[Gopal]

I cant answer that for you are you intending to connect any other devices to your internal (LAN) network? if so, then yes, you'll need some device to allow you to attach them to your LAN. If you dont, you dont even need to configure eth1 at all.

hmmm ... I might later on. not right now, my goal is to have this lab setup and be secure.

if I don't configure eth1, the sophos utm might yell at me during install.

Thanks again,

1) How do the devices connected to the eth1 on vmbr1 connect to the internet? Do I need to configure something else? like routing or something on the utm itself to have the traffic flow smoothly?
2) I tried to ssh using putty to the proxmox ve server and it didn'
t work? I might have to check /etc/ssh/sshd.conf for something ... any thoughts? same for sophos ...

 

[alexskysilk]

if I don't configure eth1, the sophos utm might yell at me during install.

I suppose I should have been more clear. you dont need to configure eth1 on proxmox. you still need to configure eth1 on the UTM.

1) How do the devices connected to the eth1 on vmbr1 connect to the internet? Do I need to configure something else? like routing or something on the utm itself to have the traffic flow smoothly?

The same as any routed configuration. the devices connect to the UTM, which in turn routes them to the internet. Sophos UTM IS a router.

I tried to ssh using putty to the proxmox ve server and it didn't work?

Think it through. you have to be able to "see" an IP address in order to access it. In other words, the initiating connection has to have a logical path from it to the target. What IP are to trying to access? what IP are you accessing it from? is there a path, physical and logical, between those two points?