proxmox talking to two different networks

O

offerlam

Renowned Member
Dec 30, 2012
224
2
83
Denmark
hi all...

ok.. so i have 3 proxmox nodes..

I have one NAS for storage of VMS, templates and so on..

I have another NAS for backup...

When doing backup i don't want to use the same network as used for storage nas... I want to conserve that nic for only reading and writing to the storage nas..

So i have my Storage nas on 10.10.99.0/24

I have my backup NAS on 10.10.98.0/24

10.10.99.0 is my management network aswell..

I want eth 1 which i don't use for anything to be able to talk to 10.10.98.0 - how do i do that? im not allowed to make a GW on the eth1 interface cause im only allowed one GW on a proxmox node.. so do i do it?

here is my network interface file..

Code: 
root@proxmox00:~# cat /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

auto eth1
iface eth1 inet static
        address  10.10.98.20
        netmask  255.255.255.0

iface eth2 inet manual

iface eth3 inet manual

auto bond0
iface bond0 inet manual
        slaves eth2 eth3
        bond_miimon 100
        bond_mode 802.3ad

auto vmbr0
iface vmbr0 inet static
        address  10.10.99.20
        netmask  255.255.255.0
        gateway  10.10.99.1
        bridge_ports bond0
        bridge_stp off
        bridge_fd 0
        bridge_maxage 0
        bridge_ageing 0
        bridge_maxwait 0
        pre-up ifconfig eth2 mtu 9000
        pre-up ifconfig eth3 mtu 9000

auto vmbr1
iface vmbr1 inet manual
        bridge_ports bond0.2
        bridge_stp off
        bridge_fd 0
        bridge_maxage 0
        bridge_ageing 0
        bridge_maxwait 0

root@proxmox00:~#
 
offerlam said:
hi all...

ok.. so i have 3 proxmox nodes..

I have one NAS for storage of VMS, templates and so on..

I have another NAS for backup...

When doing backup i don't want to use the same network as used for storage nas... I want to conserve that nic for only reading and writing to the storage nas..

So i have my Storage nas on 10.10.99.0/24

I have my backup NAS on 10.10.98.0/24

10.10.99.0 is my management network aswell..

I want eth 1 which i don't use for anything to be able to talk to 10.10.98.0 - how do i do that? im not allowed to make a GW on the eth1 interface cause im only allowed one GW on a proxmox node.. so do i do it?

here is my network interface file..

Code: 
root@proxmox00:~# cat /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

auto eth1
iface eth1 inet static
        address  10.10.98.20
        netmask  255.255.255.0

iface eth2 inet manual

iface eth3 inet manual

auto bond0
iface bond0 inet manual
        slaves eth2 eth3
        bond_miimon 100
        bond_mode 802.3ad

auto vmbr0
iface vmbr0 inet static
        address  10.10.99.20
        netmask  255.255.255.0
        gateway  10.10.99.1
        bridge_ports bond0
        bridge_stp off
        bridge_fd 0
        bridge_maxage 0
        bridge_ageing 0
        bridge_maxwait 0
        pre-up ifconfig eth2 mtu 9000
        pre-up ifconfig eth3 mtu 9000

auto vmbr1
iface vmbr1 inet manual
        bridge_ports bond0.2
        bridge_stp off
        bridge_fd 0
        bridge_maxage 0
        bridge_ageing 0
        bridge_maxwait 0

root@proxmox00:~#
Click to expand...

Hmm maybe i explained my self badly..

what i basicly want is to have my proxmox nodes connect to the storage nas on eth 3 and 4 which i have right now.. and have proxmox nodes connct to my backup nas on eth 1..

and idea how im suppose to set that up? i don't see it but it must be simple..

Thanks

Casper
 
offerlam said:
Hmm maybe i explained my self badly..

what i basicly want is to have my proxmox nodes connect to the storage nas on eth 3 and 4 which i have right now.. and have proxmox nodes connct to my backup nas on eth 1..

and idea how im suppose to set that up? i don't see it but it must be simple..

Thanks

Casper
Click to expand...

Hi,

you need only a gateway to access to externals networks.


If you have an interface with 10.10.99.20 (vmbr0-bond0), this is this interface which will be used to access to your storage nas with ip 10.10.99.0/24

BTW, you sait " eth 3 and 4", but eth3 is already bonded with eth2 ????


 
spirit said:
Hi,

you need only a gateway to access to externals networks.


If you have an interface with 10.10.99.20 (vmbr0-bond0), this is this interface which will be used to access to your storage nas with ip 10.10.99.0/24

BTW, you sait " eth 3 and 4", but eth3 is already bonded with eth2 ????
Click to expand...

Hi Spirit

and thanks for answering..

my bad about eth.. I keep forgetting im starting numeration with 0 :)

I have bond0 with eth 3 and 2 which is connected to the storage nas on 10.10.99.0/24..

in order to get at much speed as possible i don't want to both run my backup nas and storage on same nics.. since this could affect performance...

so i need eth1 on proxmox to connect to 10.10.98.0/24 in order to connect the backup nas through this nic..

that would give me full performance to storage nas on eth 3 and 2 and full performance for backup on eth 1 since all interfaces are "dedicated" to each nas

as you can see i have tried just giving eth1 ip 10.10.98.20/24 in hopes that would do it.. but my proxmox can't ping 10.10.98.11/24 which is backup nas... I suspect its because i can't put a GW on eth with 10.10.98.1 but im not sure..

basically i would like to know how i get my proxmox nodes to connect to different networks..

THANKS

Casper
 
Both the backup NAS on subnet 98 and proxmox is connected on the same switch.. I have made a sub interface in the firewall called 10.10.98.1/24

this is the output of the traceroute:

Code: 
root@proxmox00:~# traceroute -I 10.10.98.11
traceroute to 10.10.98.11 (10.10.98.11), 30 hops max, 60 byte packets
 1  10.10.98.20 (10.10.98.20)  3000.740 ms !H  3000.938 ms !H *
root@proxmox00:~#

Thanks

Casper
 
Code: 
root@proxmox00:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.10.99.0      0.0.0.0         255.255.255.0   U     0      0        0 vmbr0
10.10.98.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1
0.0.0.0         10.10.99.1      0.0.0.0         UG    0      0        0 vmbr0
root@proxmox00:~#

couldn't the problem that i uses 10.10.99.1 as gateway?
 
offerlam said:
couldn't the problem that i uses 10.10.99.1 as gateway?
Click to expand...
No, any traffic destine for 10.10.98.0/24 should be routed over eth1. Are you absolutely certain that eth1 is able to reach 10.10.98.11?
To clarify this try the following which forces routing over eth1:
traceroute -i eth1 -I 10.10.98.11
 
Hi Mir,

First here is a quick drawing of how it looks like my network... its not all of it

Udklip.JPG

I have checked the switch to make sure the lacp trunk group for the backup nas is not in any vlans... its not..

here is the network setup of the backup nas

default gateway:

Udklip1.JPG

and here the actualy network setup

Udklip3.JPG

here is the traceroute output you requestet..

Code: 
root@proxmox00:~# traceroute -i eth1 -I 10.10.98.11
traceroute to 10.10.98.11 (10.10.98.11), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  *^C
root@proxmox00:~#

also here is my ifconfig outpu for eth1

Code: 
eth1      Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:05
          inet addr:10.10.98.20  Bcast:10.10.98.255  Mask:255.255.255.0
          inet6 addr: fe80::82c1:6eff:fe64:8d05/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43566 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:1829988 (1.7 MiB)

its not much but data is being sent on the interface it seems

I also tried turning off jumbo frameson the nas since that is not setup on eth1.. did nothing..

I logged into the backup nas and tried to ping default gateway on the firewall 10.10.98.1

Code: 
Backup01> ping 10.10.98.1
PING 10.10.98.1 (10.10.98.1): 56 data bytes
64 bytes from 10.10.98.1: seq=0 ttl=64 time=0.264 ms
64 bytes from 10.10.98.1: seq=1 ttl=64 time=0.195 ms
64 bytes from 10.10.98.1: seq=2 ttl=64 time=0.202 ms
^C
--- 10.10.98.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.195/0.220/0.264 ms

Backup01> ping 10.10.98.20
PING 10.10.98.20 (10.10.98.20): 56 data bytes
^C
--- 10.10.98.20 ping statistics ---
10 packets transmitted, 0 packets received, 100% packet loss

Backup01>

So that works...

its on the proxmox side of things... or switch..

Thanks for helping out!!!

Casper
 
Code: 
eth1      Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:05
          inet addr:10.10.98.20  Bcast:10.10.98.255  Mask:255.255.255.0
          inet6 addr: fe80::82c1:6eff:fe64:8d05/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43566 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:1829988 (1.7 MiB)
Together with the results of traceroute and the above statistic I can conclude that something is wrong with eth1. The above emphasizes that no traffic is ever coming back to eth1 (RX packets:0).
 
Hi Mir and the rest of the gang..

sorry for my late respons but i had to find a day to go to the datacenter and check eth1 in relation to Mirs comment....

Ok so i configured eth1 on the two other nodes to 10.10.98.0/24 and can confirm that there are no diod blinking on eth1 on proxmox00 but the two others has blinking diods..

setting all nodes to ping 10.10.98.11 and unplugging the cables in eth1 on all nodes ment that proxmox00 and 22 lost ping.. proxmox21 kept pinging.. actually i could see some loop protection or RSTP doing its magic and than it could ping again...!?!?!? WOOOT?!!?!?

putting the cables back in eth1 on all nodes gave me replies on proxmox20 and after what looks like RSTP or loop protection magic on proxmox21 it also kept pinging ... proxmox22 never got a respons.. at anytime...

WTF is going on!!!!

but i must conclude that my interfaces are working
 
Proxmox installation. not loading after installation.

Hi Everyone, This is my first post so excuse if im doing anything wrong. Here is my situation. I have an Acer Veriton M2630G. http://www.acer.co.uk/ac/en/GB/content/professional-model-datasheet/DT.VJTEK.001 here is a link to check specs. Im installing proxmox v3.2. it installs fine everything. but as soon as it reboots it tell me no media or device. I tried to run some test of my own so i downloaded grub loader. when i use it proxmox loads and i can work. Really dont know what the problem could be, maybe grubb gets installed in the wrong partition?
 
Re: Proxmox installation. not loading after installation.

G007 said:
Hi Everyone, This is my first post so excuse if im doing anything wrong. Here is my situation. I have an Acer Veriton M2630G. http://www.acer.co.uk/ac/en/GB/content/professional-model-datasheet/DT.VJTEK.001 here is a link to check specs. Im installing proxmox v3.2. it installs fine everything. but as soon as it reboots it tell me no media or device. I tried to run some test of my own so i downloaded grub loader. when i use it proxmox loads and i can work. Really dont know what the problem could be, maybe grubb gets installed in the wrong partition?
Click to expand...

Please don't try to high jack another thread especially when the topic of that other thread has nothing related to your issue. Please create your own thread specifying your problem.

Sent from my Nexus 5
 
You must log in or register to reply here.
Top Bottom