VM network not reachable from same IP range, but reachable from others IP

[Mkz]

Hi,
I'm hurting against a wall for my VM network config.
This VM is reachable from every external IP but not from the VM in the same range IPs.
In the same time, the others VM (same host) in the same range can reachable with each other.
Just this one of them can't.
The trace which I am following is that VM get the same IP address between guest and broadcast.
I recalculated the broadcast IP on https://jodies.de/ipcalc which confirm that.

In ifconfig, broadcast value is returned "0.0.0.0".

root@prod:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 37.xxx.xxx.119  netmask 255.255.255.252  broadcast 0.0.0.0
        inet6 fe80::xxx:xxx:1973  prefixlen 64  scopeid 0x20<link>
        ether 02:00:00:xx:xx:73  txqueuelen 1000  (Ethernet)
        RX packets 2273  bytes 227015 (221.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2252  bytes 5895568 (5.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Boucle locale)
        RX packets 48  bytes 16023 (15.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 48  bytes 16023 (15.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Maybe, I've to change the VM IPs (but I don't prefer this case !)

Thanks for you're help !

 

[KevinS]

Your netmask suggests there are only 2 usable IP-Adresses in the subnet, please give a more detailed overview of your network.

 

[Mkz]

Thanks for your replay and sorry for my late answer.
I've got a range of 4 IPs : 37.xxx.xxx.116-119
This is my network details :
On the config in Proxmox : 37.xx.129.119/30 - Gateway 37.xxx.xxx.118

On the Debian VM : /etc/network/interfaces

iface lo inet loopback
auto eth0
iface eth0 inet static
        address 37.xxx.xxx.119/30
        gateway 37.xxx.xxx.118

ifconfig :

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 37.xxx.xxx.119  netmask 255.255.255.252  broadcast 0.0.0.0
        inet6 fe80::xxx:xxx:1973  prefixlen 64  scopeid 0x20<link>
        ether 02:00:xxx:xxx:19:73  txqueuelen 1000  (Ethernet)
        RX packets 17463  bytes 3612826 (3.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 13722  bytes 52819768 (50.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Do you need more info ?

 

[Mkz]

The error message :

traceroute to 37.xxx.xxx.119 (37.xxx.xxx.119), 30 hops max, 60 byte packets
connect: Permission non accordée

(translation "connect: Permission not granted")

 

[sw-omit]

A /30 network only has 2 usable IP-adresses, not 4, it is basically a point-to-point connection, meaning there is no other devices in that network then the device and the gateway.
.116 is the network-ID
.117 is the first usable IP
.118 is the second usable IP where you have your router set as
.119 is the broadcast IP (which you can't have devices set on itself)

Meaning you either need to move to a /29 (and still change IP, because for a /29 the .119 is still the broadcast-address) or go to .117 (still no other devices in the same "LAN" network.

 

[Mkz]

Thanks a lot for your explanation.
But why my 4 VM are reachable (and working well) from external IPs ?

 

[sw-omit]

I'm suspecting they might be reachable through IPv6, as IPv4 looks very strange the more I read and look back at this, at least as much as I know about it.

That said though, there are a couple of conflicting statements in this thread (perhaps cause by a language-barrier) so let's go back a step
Could you please copy the below and fill it in, replacing everything where there is an Z (the X's you can keep as-is for privacy):

Proxmox Host:
Number of (vmbr-)bridges: Z
IP-address/mask: ZZZ.XXX.XXX.ZZZ/ZZZ
Gateway: ZZZ.XXX.XXX.ZZZ

VM 1: Not reachable within host
IP-address/mask: ZZZ.XXX.XXX.ZZZ/ZZZ
Gateway: ZZZ.XXX.XXX.ZZZ

VM2: Reachable within host
IP-address/mask: ZZZ.XXX.XXX.ZZZ/ZZZ
Gateway: ZZZ.XXX.XXX.ZZZ

VM3: Reachable within host
IP-address/mask: ZZZ.XXX.XXX.ZZZ/ZZZ
Gateway: ZZZ.XXX.XXX.ZZZ

And just to confirm that I understood this correct:

• All VM's can reach the internet
• All VM's can be reached from the internet
• VM 2 and 3 can talk to eachother / ping eachother
• VM 1 can't be reached from 2/3
• VM 1 can's reach/ping 2/3

Speaking of reaching from the internet btw, how are you trying to reach them, with the IP or via a DNS-name?

 

[Mkz]

Thanks a lot for your reply ! Here are all network details on the host :

Proxmox Host:
Number of (vmbr-)bridges: 5
IP-address/mask: 54.XXX.XXX.176/24
Gateway: ZZZ.XXX.XXX.ZZZ (not found)

VM 1: Not reachable within host
IP-address/mask: 37.xxx.xxx.119/30
Gateway: 37.xxx.xxx.118
can't reach : 116
can reach : 117 - 220

VM2: Reachable within host
IP-address/mask: 37.xxx.xxx.116/30
Gateway: 37.xxx.xxx.118
can't reach : 119
can reach : 117 - 220

VM3: Reachable within host
IP-address/mask: 37.xxx.xxx.117/30
Gateway: 37.xxx.xxx.118
can't reach : 119 - 116
can reach : 220

VM4: Reachable within host
IP-address/mask: 176.xxx.xxx.220/30
Gateway: 54.xxx.xxx.254
can reach : 116 - 117 - 119 - 221

VM5: Reachable within host
IP-address/mask: 176.xxx.xxx.221/30
Gateway: 54.xxx.xxx.254
can reach : 116 - 117 - 119 - 220

• All VM's can reach the internet : YES
• All VM's can be reached from the internet : YES
• VM 2 can talk to VM3 / ping VM3
• VM 3 can't reach to VM2-VM1 / ping VM2-VM1
• VM 1 can't be reached from 2/3
• VM 1 can't reach/ping VM2
• VM 1 can reach/ping VM3

 

[sw-omit]

I have to say, that is one of the strangest and most wrong-looking setups I've seen so far...
IP's set in the network-ID/broadcast ranges, IP's where the gateway is outside of the network, network-ranges meant for point-to-point connections...

Although that might also be a lack of sleep, so I'm going to just sleep about it a bit and try to formulate a reply in the morning.
That said though, you might also want to reach out to your ISP / Network-technician / Proxmox-Hosting-Provider, as they can look at things more directly (and without privacy-masking) and maybe make more/quicker sense to it then I can, and also check if the information they have given you to use is used correctly.

Also, didn't my last question btw, are you reaching them via IP or DNS-Name from externally?

 

[Mkz]

Thanks again to walk in this maze !

Also, didn't my last question btw, are you reaching them via IP or DNS-Name from externally

Unfortunately, the problem is the same, if I reach via IP or DNS-Name :

root@prod:~# traceroute preprod.xxxx.fr

traceroute to preprod.xxxx.fr (37.xxx.xxx.116), 30 hops max, 60 byte packets

connect: Permission non accordée

root@prod:~# traceroute 37.xxx.xxx.116

traceroute to 37.xxx.xxx.116 (37.xxx.xxx.116), 30 hops max, 60 byte packets

connect: Permission non accordée

root@prod:~#

 

[sw-omit]

Are you sure that copy is from a system OUTSIDE of the proxmox-VM's?
If it is from within, I can kind of understand it, as you are asking it to perform a trace to a network-ID (.116) or broadcast (.119) IP for the range it is in, which again should not have any devices on them.

btw, from WITHIN the .117, does this command work?

ping -b 37.xxx.xxx.119

(of course replacing the xxx) and do you then get a respond directly from the 119, or only from the 118 (if any at all)

Also, even with a good night's sleep and some cups of tea, I can't really figure out how this works or why your IP-provider (whatever that is the ISP, the Network-technician or the Proxmox-Hosting-Provider) has provided you with these IP's and subnets, so I'm still suspecting there might be an error or typo somewhere in the information they send you, or that they only provided for 1 IP and by some basically miracle it works for 4... somewhat.

 

[Mkz]

Are you sure that copy is from a system OUTSIDE of the proxmox-VM's?
If it is from within, I can kind of understand it, as you are asking it to perform a trace to a network-ID (.116) or broadcast (.119) IP for the range it is in, which again should not have any devices on them.

btw, from WITHIN the .117, does this command work?

ping -b 37.xxx.xxx.119

(of course replacing the xxx) and do you then get a respond directly from the 119, or only from the 118 (if any at all)

Also, even with a good night's sleep and some cups of tea, I can't really figure out how this works or why your IP-provider (whatever that is the ISP, the Network-technician or the Proxmox-Hosting-Provider) has provided you with these IP's and subnets, so I'm still suspecting there might be an error or typo somewhere in the information they send you, or that they only provided for 1 IP and by some basically miracle it works for 4... somewhat.

First, outside of Proxmox VMs, the traceroute is working well to every IPs.

btw, from WITHIN the .117, does this command work?

ping -b 37.xxx.xxx.119

Nop :-(

And this is the list of IPs (from OVH, french provider)


Thanks again for your amazing patience !

 

[sw-omit]

Let's throw this in an entirely different direction for a moment.

While this is a very strange setup, from outside the proxmox server all the VM's are reachable, and all the VM's can reach the outside, as we figured out before. The only issue is communication between the VM's within the same server through this IP....

So why not add a second "LAN" IP to either the same port or (preferably) a second network port?
Just give the .116 an extra IP 192.168.37.116/24, the .117 gets 192.168.37.117/24, the .119 gets 192.168.37.119/24, etc. etc.
These IP's won't get routed over the public internet, so if they reach the .118 they'll just be dropped, limiting security-risks (but a seperate port restricts it even more), but since they're all in one big network, without any of them being on the broadcast or network-ID IP's, talking should be flawless.
The only issue that would arrive in the future would be if you are on multiple proxmox-nods, but then there are also way to create joined networks over WAN, or just set up a VPN-server they all connect to on an IP that they can all talk to/from, and use the internal VPN-IP's to talk to eachother.

 

[sw-omit]

Note though, since I see some are dev/pre-prod systems, and the other is production, you might want to have production and the other 2 not always connected, so maybe only turn the network-port on within Proxmox on the production-machine if you are ready to transfer something from pre-production up to production, and then turn it off again?
That way, if something that is still in development (and might have security-issues in them still) goes wrong and gets hacked for example, it can't use that back-route to gain (easier) access to your production-system, but of course you are the only one who can weigh convenience (always open) versus security (always closed/never even set up) on that one.

 

[Mkz]

Thanks a lot for your precious advices. I'll try it tomorrow !

 

[Mkz]

I give up ! I've changed the IP of the preprod VM to an another range and I can reach again the prod VM...
But the mystery remains !
Many thanks to your help @sw-omit .