connecting to proxmox ve from different subnet. Routing issue?

J

joeblow

Guest
Hi all,

I have two subnets (on a larger network of course) that I call PROD and DEV. The proxmox box is on DEV subnet. Workstations that will be working
on VMs on the proxmox box are on PROD subnet.

I have duplicated the PROD system on a 'host-only subnet' aka 'protected vlan' on
the PROXMOX box. This 'protected vlan' is set up per http://pve.proxmox.com/wiki/Protected_VLAN.

IP addresses on the virtual PROD VM's that duplicate the real PROD boxes are unchanged from their physical counterparts. Network
details of the proxmox box are at the end of this note. The vPROD VM's all interact as expected with each other, mimicking the real PROD system very nicely.

Here's the problem. WKSx workstations on the real PROD subnet need to connect to the proxmox box web interface to work with the vPROD VM's, but they
cannot. However, if I turn off the Protected_VLAN vmbr17 and reboot (not sure the reboot is necessary, but I'm following the proxmox
recommendations to reboot anytime a network change is made) then the real WKSx can connect and work with the VM's. Reactivate vmbr17 and reboot, then
the WKSx on PROD are unable to connect to proxmox. This is repeatable.

I suspect that what is happening is that when vmbr17 is active, then the return traffic to the 192.168.17 subnet is being sent to the protected
vlan, rather than out to the gateway. I have not confirmed this, and am not really sure how to. Based on my reading, this should not be behaving this way anyway.
A Protected_VLAN is a VM only construct. Is this incorrect?

Any suggestions on how to configure the proxmox box so that WKSx on PROD can connect to proxmox and work with the vPROD system?

I suppose the obvious solution is to change the ip's on all the vPROD machines to a different subnet. Rather not do that for lots of reasons, not the least of which is it'll make cloning the
PROD system to vPROD way more complicated than just the single-script fire-and-forget operation that it is now.

thanks in advance.

---------

Details:

PROD: 192.168.17.0/24 gw 192.168.17.3

DBS1 192.168.17.11

DBS2 192.168.17.12

WINDS1 192.168.17.21

WINDS2 192.168.17.22

ORCL 192.168.17.30

WKS1 192.168.17.51

WKS2 192.168.17.52

WKS3 192.168.17.53


DEV: 192.168.21.0/24 gw 192.168.21.3

PDS1 192.168.21.11

PORCL 192.168.21.30

PROXMOX 192.168.21.200

The PROXMOX on DEV is a proxmox VE box.


root@proxmox:~# cat /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

iface eth1 inet manual

iface eth2 inet manual

iface eth3 inet static
address 192.168.0.200
netmask 255.255.255.0

auto bond0
iface bond0 inet manual
slaves eth0 eth2
bond_miimon 100
bond_mode active-backup
#%09bond_mode balance-rr

auto vmbr0
iface vmbr0 inet static
address 192.168.21.200
netmask 255.255.255.0
gateway 192.168.21.3
bridge_ports bond0
bridge_stp off
bridge_fd 0

auto vmbr17
iface vmbr17 inet static
address 192.168.17.3
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0





root@proxmox:~# ifconfig
bond0 Link encap:Ethernet HWaddr e4:1f:13:30:6d:dc
inet6 addr: fe80::e61f:13ff:fe30:6ddc/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:169183 errors:0 dropped:0 overruns:0 frame:0
TX packets:100280 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:32672599 (31.1 MiB) TX bytes:25114591 (23.9 MiB)

eth0 Link encap:Ethernet HWaddr e4:1f:13:30:6d:dc
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:169183 errors:0 dropped:0 overruns:0 frame:0
TX packets:100280 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:32672599 (31.1 MiB) TX bytes:25114591 (23.9 MiB)
Interrupt:28 Memory:92000000-92012800

eth2 Link encap:Ethernet HWaddr e4:1f:13:30:6d:dc
UP BROADCAST SLAVE MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Interrupt:24 Memory:97b60000-97b80000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:316 errors:0 dropped:0 overruns:0 frame:0
TX packets:316 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:41602 (40.6 KiB) TX bytes:41602 (40.6 KiB)

venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet6 addr: fe80::1/128 Scope:Link
UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:3 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

vmbr0 Link encap:Ethernet HWaddr e4:1f:13:30:6d:dc
inet addr:192.168.21.200 Bcast:192.168.21.255 Mask:255.255.255.0
inet6 addr: fe80::e61f:13ff:fe30:6ddc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:166057 errors:0 dropped:0 overruns:0 frame:0
TX packets:99937 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:29396493 (28.0 MiB) TX bytes:24688283 (23.5 MiB)

vmbr17 Link encap:Ethernet HWaddr 12:67:19:e1:16:14
inet addr:192.168.17.3 Bcast:192.168.17.255 Mask:255.255.255.0
inet6 addr: fe80::1067:19ff:fee1:1614/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:186 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:8028 (7.8 KiB)


root@proxmox:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.21.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr0
192.168.17.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr17
0.0.0.0 192.168.21.3 0.0.0.0 UG 0 0 0 vmbr0


root@proxmox:~# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
 
joeblow said:
Hi all,

I have two subnets (on a larger network of course) that I call PROD and DEV. The proxmox box is on DEV subnet. Workstations that will be working
on VMs on the proxmox box are on PROD subnet.

I have duplicated the PROD system on a 'host-only subnet' aka 'protected vlan' on
the PROXMOX box. This 'protected vlan' is set up per http://pve.proxmox.com/wiki/Protected_VLAN.

IP addresses on the virtual PROD VM's that duplicate the real PROD boxes are unchanged from their physical counterparts. Network
details of the proxmox box are at the end of this note. The vPROD VM's all interact as expected with each other, mimicking the real PROD system very nicely.

Here's the problem. WKSx workstations on the real PROD subnet need to connect to the proxmox box web interface to work with the vPROD VM's, but they
cannot. However, if I turn off the Protected_VLAN vmbr17 and reboot (not sure the reboot is necessary, but I'm following the proxmox
recommendations to reboot anytime a network change is made) then the real WKSx can connect and work with the VM's. Reactivate vmbr17 and reboot, then
the WKSx on PROD are unable to connect to proxmox. This is repeatable.

I suspect that what is happening is that when vmbr17 is active, then the return traffic to the 192.168.17 subnet is being sent to the protected
vlan, rather than out to the gateway. I have not confirmed this, and am not really sure how to. Based on my reading, this should not be behaving this way anyway.
A Protected_VLAN is a VM only construct. Is this incorrect?

Any suggestions on how to configure the proxmox box so that WKSx on PROD can connect to proxmox and work with the vPROD system?

I suppose the obvious solution is to change the ip's on all the vPROD machines to a different subnet. Rather not do that for lots of reasons, not the least of which is it'll make cloning the
PROD system to vPROD way more complicated than just the single-script fire-and-forget operation that it is now.

thanks in advance.

---------

Details:

PROD: 192.168.17.0/24 gw 192.168.17.3

DBS1 192.168.17.11

DBS2 192.168.17.12

WINDS1 192.168.17.21

WINDS2 192.168.17.22

ORCL 192.168.17.30

WKS1 192.168.17.51

WKS2 192.168.17.52

WKS3 192.168.17.53


DEV: 192.168.21.0/24 gw 192.168.21.3

PDS1 192.168.21.11

PORCL 192.168.21.30

PROXMOX 192.168.21.200

The PROXMOX on DEV is a proxmox VE box.


root@proxmox:~# cat /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

iface eth1 inet manual

iface eth2 inet manual

iface eth3 inet static
address 192.168.0.200
netmask 255.255.255.0

auto bond0
iface bond0 inet manual
slaves eth0 eth2
bond_miimon 100
bond_mode active-backup
#%09bond_mode balance-rr

auto vmbr0
iface vmbr0 inet static
address 192.168.21.200
netmask 255.255.255.0
gateway 192.168.21.3
bridge_ports bond0
bridge_stp off
bridge_fd 0

auto vmbr17
iface vmbr17 inet static
address 192.168.17.3
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0





root@proxmox:~# ifconfig
bond0 Link encap:Ethernet HWaddr e4:1f:13:30:6d:dc
inet6 addr: fe80::e61f:13ff:fe30:6ddc/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:169183 errors:0 dropped:0 overruns:0 frame:0
TX packets:100280 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:32672599 (31.1 MiB) TX bytes:25114591 (23.9 MiB)

eth0 Link encap:Ethernet HWaddr e4:1f:13:30:6d:dc
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:169183 errors:0 dropped:0 overruns:0 frame:0
TX packets:100280 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:32672599 (31.1 MiB) TX bytes:25114591 (23.9 MiB)
Interrupt:28 Memory:92000000-92012800

eth2 Link encap:Ethernet HWaddr e4:1f:13:30:6d:dc
UP BROADCAST SLAVE MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Interrupt:24 Memory:97b60000-97b80000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:316 errors:0 dropped:0 overruns:0 frame:0
TX packets:316 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:41602 (40.6 KiB) TX bytes:41602 (40.6 KiB)

venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet6 addr: fe80::1/128 Scope:Link
UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:3 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

vmbr0 Link encap:Ethernet HWaddr e4:1f:13:30:6d:dc
inet addr:192.168.21.200 Bcast:192.168.21.255 Mask:255.255.255.0
inet6 addr: fe80::e61f:13ff:fe30:6ddc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:166057 errors:0 dropped:0 overruns:0 frame:0
TX packets:99937 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:29396493 (28.0 MiB) TX bytes:24688283 (23.5 MiB)

vmbr17 Link encap:Ethernet HWaddr 12:67:19:e1:16:14
inet addr:192.168.17.3 Bcast:192.168.17.255 Mask:255.255.255.0
inet6 addr: fe80::1067:19ff:fee1:1614/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:186 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:8028 (7.8 KiB)


root@proxmox:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.21.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr0
192.168.17.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr17
0.0.0.0 192.168.21.3 0.0.0.0 UG 0 0 0 vmbr0


root@proxmox:~# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Click to expand...
Hi,
due to your protected vlan you can't reach any real hosts on this network!
How should the network be reachable without nic? Look with "ip route" and you see that 192.168.17.0/24 is reachable over vmbr17 - but only on this host without real NIC.

Why you don't use eth1 for vmbr17? If you have VMs which needs IPs from vmbr17 and they should accessible from the outside, the must be in the real netwaork.

Udo
 
Actually, I expect and want that the vm's on the protected_vlan vmbr17 should -not- be able to get out of the proxmox host. This is expected and desired behaviour. That way there is no inadvertant traffic between the real PROD network and the virtual vPROD network.

What I do expect is that a web browser on a real WKS on real 192.168.17.5x should be able to (assuming proper routing in the real network, which is a good assumption in this case) connect and interact with https://192.168.21.200 (aka https://proxmox) since 192.168.21.200 is assigned to vmbr0, which -is- connected to a real nic (eth0/eth2 via bond0) .

What am I not getting here?

Thanks!
 
joeblow said:
Actually, I expect and want that the vm's on the protected_vlan vmbr17 should -not- be able to get out of the proxmox host. This is expected and desired behaviour. That way there is no inadvertant traffic between the real PROD network and the virtual vPROD network.

What I do expect is that a web browser on a real WKS on real 192.168.17.5x should be able to (assuming proper routing in the real network, which is a good assumption in this case) connect and interact with https://192.168.21.200 (aka https://proxmox) since 192.168.21.200 is assigned to vmbr0, which -is- connected to a real nic (eth0/eth2 via bond0) .

What am I not getting here?

Thanks!
Click to expand...
Hi,
that's quite easy. Simply don't use an IP on the bridge vmbr17! You only need an IP on a bridge, if the host should accessible on this IP. If you use something like this:
Code: 
auto vmbr17
iface vmbr17 inet manual
    bridge_ports none
    bridge_stp off
    bridge_fd 0
your clients from the 192.168.17.0/24 network can reach your host via the default router.

Udo
 
Perfect! And perfectly obvious in retrospect. Thanks for clearing the fog!
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back