2 NIC behind pfSense

[apmuthu]

A ProxMox 1.3 server's eth0 is connected to the LAN socket of a pfSense Router (physical hardware unit) that provides a DHCP. The pfSense's WAN is connected to a PPPoE Cable Modem that provides a global IP. The eth1 of the ProxMox server is connected to the office LAN switch. An SMEServer v8.0 beta 4 (CentOS v5.3 based) is run as a KVM inside the Proxmox server taking it's WAN from vmbr0 (bridged to eth0) and issuing DHCP on vmbr1 (bridged to eth1).

Internet => WAN => pfSense => LAN => eth0 => ProxMox => vmbr0.eth0 => SMEServer KVM => vmbr1.eth1 => OfficeLAN => OfficeMachines

We need to prevent the vmbr1 from getting a DHCP IP from the pfSense and instead possibly take it from the SMEServer KVM. Since the KVM is not running while the host is being booted, we allot a static IP on the SMEserver's LAN subnet.

The following /etc/network/interfaces file on the ProxMox host does not do the job:-

# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

iface eth1 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.12.178
netmask 255.255.255.0
gateway 192.168.12.1
bridge_ports eth0
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 10.20.30.30
netmask 255.255.255.0
bridge_ports eth1
bridge_stp off
bridge_fd 0

There seems to be no gateway entry for vmbr1 in the above - it should actually be 10.20.30.1 for the SMEServer to handle requests for the OfficeLAN.

I also found that the files in /proc/sys/net/bridge/ were all 1 and made them all 0 and rebooted thus:

echo 0 > /proc/sys/net/bridge/bridge-nf-call-arptables
echo 0 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
echo 0 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 0 > /proc/sys/net/bridge/bridge-nf-filter-pppoe-tagged
echo 0 > /proc/sys/net/bridge/bridge-nf-filter-vlan-tagged

There is no internet connection available from the OfficeLAN side. The SMEServer cannot ping out anywhere from within and viceversa.

The SMEServer has been configured to use e1000 NICs for both eth0 and eth1 from within it - linking to vmbr0 and vmbr1.

The /etc/resolv.conf inside the SMEServer KVM was found to contain only the static LAN IP allotted to the OfficeLAN Gateway 10.20.30.1. Added 192.168.12.1 and two other external DNS Server IPs as well to it and did a
/etc/init.d/network restart on the SMEServer.

 

[apmuthu]

The problem is that there is no "broadcast= " line when ProxMox creates bridges (vmbr#) or devices (eth#).

The command ip link show provides some insight.

The Debian Network Howto states that there should be the following entities:

# The loopback network interface 
auto lo 
iface lo inet loopback  

# The primary network interface 
auto eth0 
iface eth0 inet static  
address 192.168.1.90  
gateway 192.168.1.1  
netmask 255.255.255.0  
network 192.168.1.0  
broadcast 192.168.1.255

In my setup, under the vmbr1 stanza, I needed to put in:

broadcast=10.20.30.255

and it now works.

Possible values for the iface eth0 inet line without the lines below it are:

iface eth0 inet manual

iface eth0 inet dhcp

WindowsXP KVMs, however, are getting a
DHCP IP 10.0.2.15 from a
DHCP Server 10.0.2.2 with
DNS Server 10.0.2.3 and
Gateway 10.0.2.2
possibly from the qemu-server. It is visible only in KVMs where we choose to have a network card that is of NAT type.
There seem to be a few posts on this issue - 1 2 .

The console command ip route on the host provides the actual network config that the client machines will get.

 

[apmuthu]

Re: 2 NIC behind pfSense - solved

Got the issue sorted out and posted it in the Wiki.

 

[tesquenure]

Hi Apmuthu,

Nice WIKI, i try it soon.
And try it for an PfSense on a VM.

I use SME 7.4 and i have some trouble with reboot in the VM.
and you ?
http://www.proxmox.com/forum/showthread.php?t=2129
http://www.proxmox.com/forum/showthread.php?t=2100

Im interest to echange about SME on proxmox.

tesquenure (AT) gmail.com

Regards
Tesquenure