How to connect 2 KVMs on different networks (Communication issue)

[tico46]

Hey guys, I'm just at a point that I'm clueless and don't know what else to try. I feel like I've gotten closer and closer but I can't troubleshoot this any further! I know I'm SO close, but just STUMPED ! Here's how my network looks like:

My goal is to setup two networks: 192.168.1.0 and 192.168.200.0, both windows server 2008 Active directory server. Trying to create trusted relationship (but can't ping one another).

To do this, I have a desktop machine that I'm running proxmox on with 2 nics and 2 routers. eth0 is bridged with vmbr0 and eth1 with vmbr1. I have a cat5 cable connected from router 1 switch port (main router/192.168.1.1 gw) to router 2 (2nd router/192.168.200.1 gw) switch port. So far so good, as I can ping both gateways from the proxmox shell. Here's the problem: From the server 2k8 KVMs, I can ping their respective gateways, but I can't ping 192.168.1.1 gateway from 192.168.200.0 domain and can't ping 192.168.200.1 gateway from 192.168.1.1 domain. It's driving me crazy! Get error "Pinging 192.168...Destination net unreachable" so it does see it, but there's no route.

I would think that if I could ping both gateways from the proxmox shell, that I'm good to go. Apparently not. Not sure if this is a proxmox issue or a server issue. Any help would be greatly appreciated as I've been working on this for 3 days and made sure that all my options were exhausted (forum search, google search, troubleshooting etc) before I posted on the forum.

Thanks!

 

[cesarpk]

For the Host see this link the tittle "Multiple IP addresses on One Interface":
https://wiki.debian.org/NetworkConf...erface#Multiple IP addresses on One Interface
... and apply it to vmbr0 if you have only a NIC

And for the guest (Windows VM), if you have only a NIC into your Host, you can do it of two manners:
A) Assign a second IP since the IP network config of your Windows Server (Obviously with the other subnet), or
B) For your guest (Windows VM), you can create 2 virtual NICs linked to vmbr0, and the second virtual NIC of your VM must to have configurated a IP address of the second subnet.

But if you have two NiCs on the PVE Host: Then the VM must have two virtual NICs, and these virtual NICs must to have configurated the differents subnets that you need

 

[m.ardito]

I would think that if I could ping both gateways from the proxmox shell, that I'm good to go. Apparently not.

you have to remember that 192.168.1.0/24 and 192.168.200.0/24 are two separated networks. They are able to reach only hosts in their networks. That's without a router between networks.

even if pve "knows" both networks, your vm don't.
pve has two nics, and reaches both networks, while each vm has only one and talks to only one network.

but you could simply (eg) add a second virtual nic to each vm, with the right IP and bridge (vmbr0/1) in order to make the two vm find themselves...

Marco

 

[tico46]

you have to remember that 192.168.1.0/24 and 192.168.200.0/24 are two separated networks. They are able to reach only hosts in their networks. That's without a router between networks.

even if pve "knows" both networks, your vm don't.
pve has two nics, and reaches both networks, while each vm has only one and talks to only one network.

but you could simply (eg) add a second virtual nic to each vm, with the right IP and bridge (vmbr0/1) in order to make the two vm find themselves...

Marco

Marco, right I understand that they are on two different networks. What I'm essentially trying to do is create a lab in which I have one network with a web server and I'm trying to get another network to touch that web server. In order to do that, I at least have to ping the web server, which I can't cause there's no connectivity outside the network of course.

I mean the way I understand it, is that for us to access a website, that website has an IP that is on a different network than ours but somehow, our packets get routed to that web server. I have two routers right now, router 1 is connected to router 2. Just wish there was an easier way to send packets from NIC 1, through Router 2 (192.168.200.1 gateway) through router 1 (192.168.1.1) back through router 2 to NIC 2. This is what my network looks like

NIC1 --> Router 2
NIC2 --> Router 2
Router 2 --> Router 1

Like I said when I ping the gateway 192.168.200.1 from the 192.168.1.0 network, i get Destination net unreachable. I understand this means that it sees the router, but it doesn't have a route. Don't know how to make that route!

Thanks for the help!

 

[m.ardito]

I have two routers right now, router 1 is connected to router 2.

that is not sufficient: your "routers" have to be configured to route those address.. and remember also that you are using private ip addresses (ie: non routable)...

don't take it bad, but imho you have to better understand networking basic before playing with pve and virtualization.
this has nothing to do with pve, which is a high level technical tool aimed at virtualization in a production environment, it is not a network learning tool...

Marco

 

[tico46]

Ok I understand that this may not be possible with my current equipment (2 consumer grade routers) as it is not possible to forward packets to another gateway that is on a different network.

After researching all day today, I have come to the realization that in order to do this, I would need a managed switch to forward packets to different networks (by creating vLANS).

Can anyone confirm this? I would like to mark as "solved"....Thanks!

 

[pirateghost]

You should be able to accomplish what you want if your routers support adding a static route(a lot of them I have seen allow this). This really is just a basic networking concept though and has nothing to do with proxmox.

You could learn about basic routing and networking and virtualize a router and get exactly what you want.

Sent from my Nexus 5

 

[cesarpk]

Hi tico46

Why do you don't make that i say?... if you do it, you will see that will work well
See my previous post and do it (I have it re edited)!!!!
http://forum.proxmox.com/threads/16...works-(Communication-issue)?p=85655#post85655

 

[cesarpk]

You should be able to accomplish what you want if your routers support adding a static route(a lot of them I have seen allow this). This really is just a basic networking concept though and has nothing to do with proxmox.

Excuse me pirateghost, but if the Routers aren't in the same subnet for that both can see between they, then the Routing table will not work.

And I agree that the original question has nothing to do directly with the use of PVE

 

[pirateghost]

But his routers are in LANs. That means they are controllable and able to add static routes and see each other

Sent from my Nexus 5

 

[cesarpk]

But his routers are in LANs. That means they are controllable and able to add static routes and see each other

Please, excuse me again pirateghost, but if the router aren't in the same subnet, never will can see to each other, and if will never can see to each other, then the routing table will not work. For that this works, always the routers need see to the other across the same subnet.

 

[pirateghost]

Please, excuse me again pirateghost, but if the router aren't in the same subnet, never will can see to each other, and if will never can see to each other, then the routing table will not work. For that this works, always the routers need see to the other across the same subnet.

routers SHOULD NOT be in the same subnet...thats not how it works. if they were in the same subnet, they wouldnt need a route because they would be in the same subnet!

i have many VLANs...they are NOT in the same subnet, because, well, thats the purpose of VLANs to separate subnets.
in order to get the VLANs to talk to each other, i have ROUTES in place so they can see each other.

based on THIS POST the OP is trying to talk to a webserver (so, A, he could just forward port 80 through the router) but in his explanation he says ROUTER 2 is CONNECTED TO ROUTER1, this means the 2 routers can talk to each other. he would need to get crafty with his config, but it is possible to make router 2 push traffic to router 1 by a static route.

static route 192.168.2.0/24 next-hop 192.168.1.1 (or however the router would allow him to configure)

 

[cesarpk]

NIC1 --> Router 2
NIC2 --> Router 2
Router 2 --> Router 1

As i understand, this is your scenery:

NIC1 (with Subnet 1) ---> NIC3-Router 2-NIC4 (with Subnet 2) ---> NIC5-Router 1-NIC6 (with Subnet 3) ----> To Internet
NIC2 (with Subnet ???) -> NIC3-Router 2-NIC4

And the Windows Servers are:
"Windows Server-1" is in the Subnet-1, and
"Windows Server-2" is in the Subnet-2,

Right?
Your Routers are PCs, Home Routers, Professional Routers, or what?

If This is correct, for that Windows Server-2 can initiate a connection TCP/IP in the Subnet 1, the Router 2 should permit connections since the Subnet 2 to the Subnet 1, else the connection will not possible, generally little Routers for the home have a firewall activate and not allowed initiate connections of both sides, others Routers may be that have DMZ or Virtual Server configurations for get the reverse comunication.

If your scenery is other, will be better that you do a graph of all your scenery for understand better, including:
1- The physical connectios
2- The Subnets
3- The IPs of everything
5- Tell us if your Routers are Hardware Routers (with Brand and model), PCs, or what.

I believe that with this information, much people can help you

 

[udo]

Marco, right I understand that they are on two different networks. What I'm essentially trying to do is create a lab in which I have one network with a web server and I'm trying to get another network to touch that web server. In order to do that, I at least have to ping the web server, which I can't cause there's no connectivity outside the network of course.

I mean the way I understand it, is that for us to access a website, that website has an IP that is on a different network than ours but somehow, our packets get routed to that web server. I have two routers right now, router 1 is connected to router 2. Just wish there was an easier way to send packets from NIC 1, through Router 2 (192.168.200.1 gateway) through router 1 (192.168.1.1) back through router 2 to NIC 2. This is what my network looks like

NIC1 --> Router 2
NIC2 --> Router 2
Router 2 --> Router 1

Like I said when I ping the gateway 192.168.200.1 from the 192.168.1.0 network, i get Destination net unreachable. I understand this means that it sees the router, but it doesn't have a route. Don't know how to make that route!

Thanks for the help!

Hi,
your network config can't be right (or I miss something).
For me something like this makes sense:
NIC1 -> Router 1 (192.168.1.1/24)
NIC2 -> Router 2 (192.168.200.1/24)
Router 2 -> Router 1 (like 192.168.5.2 -> 192.168.5.1/30 or something else)

E.G. hosts on 192.168.1.0/24 has as default gateway 192.168.1.1, and hosts on 192.168.200.0/24 192.168.200.1.

Perhaps you have the problem, that windows firewall don't allow to ping the host??
You can simply test with tcpdump - ping from the VM1 to VM2 and use "tcpdump -i vmbr0 host IP.OF.NODE.ONE" to see the ping-request.
If you see the same request on vmbr1 the routing is fine!

If you do NAT between the Network you have perhaps another problem?


Udo

PS: I hope you used brideg networking for your VMs?? Please post your VM-configs

 

[cesarpk]

If you do NAT between the Network you have perhaps another problem?
Udo

Hi udo, a great master of this forum. or at least for me .
It is a pleasure for me to greet you.

And please, let me make a clarification:

I am absolutely sure that if tico46 will use NAT in the Router, will have problems, why?...

Because if "Windows-Server-1" initiates communication to "Windows-Server-2" for sync Active Directory, "Windows-Server-1" will find the real IP address of the "Windows-Server-2", and all communication will be good.

But if "Windows-Server-2" want to start the sync of Active Directory with "Windows-Server-1", "Windows-Server-1" will find other IP address due to technologies such as "NAT" or "MASQUERADE" that are in between (by rules in the router), being well "Windows-Server-1" will reject the synchronization due to that the IP address of source is other.

In conclusion technologies such as "NAT" or "MASQUERADE" should not be used for this type of communications due to is necessary a communication bilateral and being starting at any of both sides and with their real IP addresses for the recipient.

Only as comment: With Linux and ruies of firewall using "iptables", if the "Router-1" is Linux, will be necessary use rules as "PREROUTING", "DNAT" and "FORWARD" for get a communicarion between both "Windows Servers" without problems.
This technique always worked very well for me.

But if tico46 have as routers home routers, he should configure it if the router have the required option.

And for me the problem is in the correct configuration of his routers.

Best regards
Cesar

 

[cesarpk]

static route 192.168.2.0/24 next-hop 192.168.1.1 (or however the router would allow him to configure)

Hi pirateghost

Please, let me to do a question:

If Router-1 have 2 subnets = one for the LAN and the other for the wAN, and
If Router-2 have 2 subnets = one for the LAN (that is same of the WAN of Router.-1), and the second for the wAN ....

Then, the IP address WAN of the Router-1 is in the same subnet of the IP address LAN of the Router-2?

Best regards
Cesar