2 separate sub-nets 1 NIC

[bstriker]

I've searched this topic for awhile now and can't seem to find any solution. Everyone seems to have different things working for them.
And with this being "an-almost-production-server" I can't really afford to "tinker" too much..

What I have is a Proxmox host with 1 NIC, 2 seperate subnets. (2 separate CIDR blocks)
Default Install with vmbr0 being bridged over eth0
vmbr0 has a /29 statically configured with a gateway
VZ / KVM can use the bridge vmbr0 or use venet, no problem.

I've created another vmbr1 with the /28 statically assigned.
I can ping the vmbr1 address no problem
but other VM's can't get out when assigned an IP in that range.

My understanding of networking is horrid, or its my understanding of interfacing / configuration.
Either way, can someone please put me out of misery and point me in the right direction
Thanks in advance

/etc/networking/interfaces:

auto lo
iface lo inet loopback


auto eth0
iface eth0 inet manual

#XXX.XXX.99.0/29 (usable range: 2-6)
auto vmbr0
iface vmbr0 inet static
        address XXX.XXX.99.2
        netmask 255.255.255.248
        gateway XXX.XXX.99.1
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
        #The line following did not work
        #post-up route add -net XXX.XX.127.130 netmask 255.255.255.240 dev vmbr0

#XXX.XX.127.128/28 (usable range: 130-142)
auto vmbr1
iface vmbr1 inet static
        address XXX.XX.127.130
        netmask 255.255.255.240
        gateway XXX.XX.127.129
        metric 100
        bridge_ports none
        bridge_stp off
        bridge_fd 0

 

[mir]

The answer is simple: A nic can only have one gateway. For your requirements two nics are required.

 

[bstriker]

The answer is simple: A nic can only have one gateway. For your requirements two nics are required.

with all respect that's not entirely true with my findings.
#1 metric 100 sets the priority lower.
#2 I've witnessed what I'm trying to achieve working. Just intermittent and leaving me going "wht da fuq" like a gameboy working without batteries or gun not firing just when you thought you loaded it correctly. I'm almost bald because of this matter

I've playing with
ip route add XXX.XX.127.128/28 dev vmbr0

and for some reason venet assigned ips are even more flaky....

 

[mir]

Sure, you can do what you like, but it will never work.

 

[Nemesiz]

2 nic - 2 ISP

ISP 1 - 1 nic - 3 IP via DHCP ( 2 ip in x.x.85.0/24 subnet and 1 in x.x.84.0/24)
ISP 2 - 1 nic - 1 IP via DHCP ( x.x.0.0/18 subnet )

# ip route
192.168.11.0/30 dev tun2 proto kernel scope link src 192.168.11.1
10.10.16.0/24 dev tun10 proto kernel scope link src 10.10.16.1
x.x.85.0/24 dev vmbr1v1 proto kernel scope link src x.x.85.y1
x.x.85.0/24 dev vmbr1v2 proto kernel scope link src x.x.85.y2
x.x.84.0/24 dev vmbr1 proto kernel scope link src x.x.84.z
10.10.8.0/21 dev vmbr0 proto kernel scope link src 10.10.8.1
x.x.0.0/18 dev vmbr2 proto kernel scope link src x.x.x.x
default
nexthop via x.x.84.1 dev vmbr1 weight 1
nexthop via x.x.85.1 dev vmbr1v1 weight 1
nexthop via x.x.85.1 dev vmbr1v2 weight 1
nexthop via x.x.63.254 dev vmbr2 weight 1


# ip rule
0: from all lookup local
1: from 10.10.10.2 fwmark 0x2 lookup ddos2
32756: from 10.10.10.8 lookup mv3
32757: from 10.10.8.0/21 lookup mv1
32758: from all to x.x.x.x lookup teo
32759: from x.x.x.x lookup teo
32760: from all to x.x.85.y2 lookup mv3
32761: from x.x.85.y2 lookup mv3
32762: from all to x.x.85.y1 lookup mv2
32763: from x.x.85.y1 lookup mv2
32764: from all to x.x.84.z lookup mv1
32765: from x.x.84.z lookup mv1
32766: from all lookup main
32767: from all lookup default

cat /etc/iproute2/rt_tables
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep

10 mv1
11 mv2
12 mv3
20 teo
98 ddos2
99 ddos1

cat /etc/network/interfaces

.....
auto eth0
iface eth0 inet manual
.....
auto eth2
iface eth1 inet manual
.....
auto vmbr1
iface vmbr1 inet manual
post-up /root/bin/dhcp/rules_eth $IFACE mv1 up
post-down /root/bin/dhcp/rules_eth $IFACE mv1 down
bridge_ports eth0
bridge_stp off
bridge_fd 0


auto vmbr1v1
iface vmbr1v1 inet manual
pre-up ifup vmbr1 2> /dev/null
pre-up ip link add link vmbr1 name $IFACE address x:x:x:x:x:x type macvlan 2> /dev/null
up ifconfig $IFACE 0.0.0.0 up
down ifconfig $IFACE down
post-up /root/bin/dhcp/rules_eth $IFACE mv2 up
post-down /root/bin/dhcp/rules_eth $IFACE mv2 down
post-down ip link del link dev $IFACE 2> /dev/null


auto vmbr1v2
iface vmbr1v2 inet manual
pre-up ifup vmbr1 2> /dev/null
pre-up ip link add link vmbr1 name $IFACE address x:x:x:x:x:x type macvlan 2> /dev/null
up ifconfig $IFACE 0.0.0.0 up
down ifconfig $IFACE down
post-up /root/bin/dhcp/rules_eth $IFACE mv3 up
post-down /root/bin/dhcp/rules_eth $IFACE mv3 down
post-down ip link del link dev $IFACE 2> /dev/null


auto vmbr2
iface vmbr2 inet manual
post-up /root/bin/dhcp/rules_eth $IFACE teo up
post-down /root/bin/dhcp/rules_eth $IFACE teo down
bridge_ports eth2
bridge_stp off
bridge_fd 0

I had problem with DHCP so I had to make it static (all IP are static but ISP wants to register it by DHCP, so i made some scripts to send and receive DHCP request and to check IP settings.

You can try to use macvlan to clone bridge with different MAC or just make virtual network alias with static IP settings.

 

[bstriker]

Sure, you can do what you like, but it will never work.

well its working now, i had a typo with my post-up route add line
post-up route add -net XXX.XX.127.128 netmask 255.255.255.240 dev vmbr0

as far as the second gw, idk if its a placebo effect or what. I got what I wanted working.

 

[wahmed]

The answer is simple: A nic can only have one gateway. For your requirements two nics are required.

I have to agree with this. 1 NIC, 1 Gateway. Period.

Gateway address is tighted up with IP address assigned to that physical NIC. Without assigning 2nd IP address for the 2nd gateway, NIC simply wont look for the gateway. And it is a fact, 1 NIC cannot have multiple IP addresses.
If it does work, i would really want to know. Will be a great learning experience.

 

[mir]

A gateway acts like a catch-all for a net. If a requested ip is not belonging to the net the gateway will receive the request for further routing. If a nic has two gateways assigned how can you be sure that ip's are routed correctly? The used gateway on a given net will choose the first configured gateway when routing is required and there is now guaranty that the gateways will always have the same order. The order is simply determined by which net is configured first.

 

[screenie]

He could create a second routing table for the second network and perform source routing to the gateway on this net;

 

[bstriker]

A gateway acts like a catch-all for a net. If a requested ip is not belonging to the net the gateway will receive the request for further routing. If a nic has two gateways assigned how can you be sure that ip's are routed correctly? The used gateway on a given net will choose the first configured gateway when routing is required and there is now guaranty that the gateways will always have the same order. The order is simply determined by which net is configured first.

This makes a lot of sense. Thanks a lot guys for the networking 101 or 201