How can I configure Proxmox and pfSense vm?

Discussion in 'Proxmox VE: Networking and Firewall' started by eiger3970, Jan 8, 2019.

  1. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    I followed this guide https://www.netgate.com/docs/pfsense/virtualization/virtualizing-pfsense-with-proxmox.html to setup the Proxmox VM pfSense, however I think the LAN and WAN are mixed up with the vmbr0, vmbr1, vmbr2 and the port/slaves eth0, eth1 and eth2?

    The setup:
    Proxmox host:
    eth0 Network Device. (physical NIC, not sure if LAN or WAN).

    eth1 Network Device. (physical NIC, not sure if LAN or WAN).

    vmbr0 Linux Bridge
    Port/slave: eth0
    inet addr:192.168.1.140 Bcast:192.168.1.255 Mask:255.255.255.0 Gateway:192.168.1.170.

    vmbr1 Linux Bridge
    Port/slave: eth1.
    This is the Proxmox host’s physical NIC WAN, that the ISP’s modem connects to.

    vmbr2 Linux Bridge
    Port/slave: eth2
    inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0.
    This is the Proxmox host’s physical NIC LAN, that the 24 port switch connects to.

    LAN PC connected to the switch:
    inet addr:192.168.1.120 Bcast:192.168.1.255 Mask:255.255.255.0.

    Some testing results:
    LAN PC 192.168.1.120 could ping Proxmox host 192.168.1.140.
    LAN PC could ssh into Proxmox host.
    LAN PC could not scp into Proxmox host.

    So I scp’ed from the Proxmox host to the LAN PC to pull the pfSense.iso image.

    On Proxmox, created VM and installed pfSense like a charm.
    However, unclear how to connect LAN PC to pfSense GUI and Internet?
    I can only view pfSense welcome screen (black and white shell) in Proxmox VM pfSense console.
    LAN PC can ping 192.168.1.1, but cannot browse the GUI to configure pfSense.
     
  2. elurex

    elurex Member
    Proxmox VE Subscriber

    Joined:
    Oct 28, 2015
    Messages:
    137
    Likes Received:
    3
    make sure you use vNIC e1000 and not virtio vNIC
     
  3. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Thanks, I might try that.
    I noticed the setup guide is unclear and is actually for 3 NICs, however the guide suggests it is for 2 NICs which is what my setup is.
     
  4. hodo

    hodo New Member

    Joined:
    Jun 22, 2016
    Messages:
    28
    Likes Received:
    4
    Hello eiger3970,

    make sure you have set the following settings:

    1. You need 1 vNIC in the pfsense VM for WAN connected to vmbr1.
    2. You need 1 vNIC in the pfsense VM for LAN connected to vmbr2.
    3. give your pfsense VM on the LAN-Side a free static IP in the 192.168.1.0/24 network. (Maybe the 192.168.1.2)
    4. Let your pfsense VM connect on the WAN side to your modem. (With your informations given, i cant suggest you anything here.)
    5. For internet access set the default gateway of your LAN-Machines to your pfsense LAN static ip. (maybe 192.168.1.2)
    6. Set your firewall and routing rules in the pfsense VM (Webinterface)
    7. I dont get why you need your vmbr0. (Maybe more information is needed)

    With your actual config, the proxmox host (incl. the webgui on :8006) is accessible on the 192.168.1.1 and the 192.168.1.140 address.

    If you want your pfsense lan on 192.168.1.1, give me a short hint, and i can give you the configuration steps needed.

    Sidenote: I have virtio vNICs in my pfsense vm for over 2 years now. I think you have to disable the offload functions in the pfsense webgui and can use the virtio vNICs without problems.

    kind regards,
    Hodo
     
  5. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Thanks Hodo, a great help.
    1. and 2. are already set like that.
    3. has IP 192.168.1.1.
    4. I assume it’s correct, as per the Netgate guide.
    5. and 6. will be set when I can access the vm pfSense GUI.
    7. as per Netgate guide, vmbr0 is reserved for Proxmox admin. However, I think this requires 3 NICs. I think 3 NICs allow access to Proxmox to fix cm pfSense if it fails at some stage. This is my concern with 2 NICs...how to access pfSense if it fails?
     
  6. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Elurex, I tried e1000 on vmbr0 and vmbr1, however error: Parameter verification failed. (400)
    net0: hotplug problem - error on hot-unplugging device ‘net0’
    I decided to revert this attempted change, as I think Hodo’s reply is considering I have 2 NICs, not 3.
     
  7. hodo

    hodo New Member

    Joined:
    Jun 22, 2016
    Messages:
    28
    Likes Received:
    4
    Hello eiger3970,

    your reply let me assume, that you want to have the pfsense VM with 192.168.1.1 on the lan side.

    Actual you have given the 192.168.1.1 address to the proxmox host. (on vmbr2) - not the pfsense vm

    What you need to do:

    1. edit your proxmox network config and empty the address and netmask fields on your vmbr0
    2. edit your proxmox network config and set the 192.168.1.140 address, netmask 255.255.255.0 fields on your vmbr2
    3. reboot
    3. check your pfsense virtual hardware config -> 1 vNIC on vmbr1 (WAN), 1 vNIC on vmbr2 (LAN) , note your mac addresses on these 2 interfaces
    4. start the pfsense vm, in the vm-console there is an option to assign the network interfaces (At first boot the installer will ask you about these)
    5. set WAN interface to the vtnet adapter with the corresponding macaddress of the vNIC on vmbr1
    6. set LAN interface to the vtnet adapter with the corresponding macaddress of the vNIC on vmbr2
    7. in the vm-console give the LAN interface the static ip setting of 192.168.1.1, netmask 255.255.255.0

    At this point you should point your browser to the 192.168.1.1 webaddress and you can start configuring the pfsense VM via webgui. (Setting the modem dialup, gateways, dhcp....)

    May i ask what the 192.168.1.170 gateway is in your config?

    As i said, the virtio driver in pfsense has matured. There is no need to stick to e1000.

    kind regards,
    hodo
     
  8. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Thank you, that really clarified what I was learning to do, however needed assistance with.
    Will try when on location.
    192.168.1.170 was the hardware pfSense router’s default gateway address.
    I changed the default gateway from 192.168.1.1 to 192.168.1.70, to alphabetically order my network topology.
    E.g. Proxmox 192.168.1.140, Router 192.168.1.170 etc.
     
  9. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Actually, vmbr0 has 192.168.1.140 for Proxmox.
     
  10. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Ok, I did steps 1-3 and now I cannot ping or browse Proxmox on 192.168.1.140?
     
  11. hodo

    hodo New Member

    Joined:
    Jun 22, 2016
    Messages:
    28
    Likes Received:
    4
    Hello eiger3970,

    you said vmbr2 is on eth2 and eth2 is connected to your LAN Switch. Make sure your client is connected to the lan switch and you have an ip in the 192.168.1.0/24 subnet.

    With this settings set, you should access your proxmox host on 192.168.1.140.

    kind regards
    hodo
     
  12. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Yes, but vmbr2 with port/slave eth2 is pfSense, which is newly installed, but no GUI configuration yet. I’m stuck at post7, as Proxmox shows vmbr2 with IP address 192.168.1.1 and Subnet 255.255.255.0 and no gateway.
    Now I’m trying to regain GUI access to Proxmox somehow? I have terminal access, but not sure what to set /etc/network/interfaces to?
     
  13. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Proxmox ifconfig shows
    eth1, eth0, ap145i0, vmbr0 and vmbr1.
     
  14. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    To be clear, eth2 is a virtual port, not a real physical NIC.
     
  15. hodo

    hodo New Member

    Joined:
    Jun 22, 2016
    Messages:
    28
    Likes Received:
    4
    Hello eiger3970,

    okay, then you can try to revert the changes you made to vmbr0.

    In /etc/network/interfaces look for the line:

    iface vmbr0 ....


    set the following:

    iface vmbr0 inet static
    address 192.168.1.140
    netmask 255.255.255.0
    gateway 192.168.1.170

    -> reboot

    Then we could start again. But this time i need the respone of "ip a" as root.

    In your first post you said, that eth2 is connected to your physical LAN Switch. It looks like your "real" LAN access is on eth0, right?

    Then vmbr0 is your LAN-Bridge. Then you should connect your pfsense-vm LAN vNIC to vmbr0 instead of vmbr2.

    kind regards,
    hodo
     
  16. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Yes, thank you, I see the mistake.
    I think as I was following the Netgate guide for 3 NICs, I understood the guide as 2 NICs which is my setup.
    I should have worded vmbr2 as ‘to become the LAN NIC’.

    So, I have regained GUI access to Proxmox via:
    /etc/network/interfaces
    auto vmbr0
    iface vmbr0 inet static
    address 192.168.1.140
    netmask 255.255.255.0
    gateway 192.168.1.170
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

    This took forever to reconnect as vmbr0 showed bridge_ports none, rather than bridge_ports eth0.
    I also have the Proxmox server plugged into the office room next to my LAN PC, for the VGA monitor connection to Proxmox terminal.

    I will now move the Proxmox server to the ISP modem room and connect the modem into the Proxmox server’s NIC2, which I think is eth1.

    Will then add ip a output.

    Is there a preference/rule for which eth0 or erh1 is LAN and WAN?
     
  17. hodo

    hodo New Member

    Joined:
    Jun 22, 2016
    Messages:
    28
    Likes Received:
    4
    Hello eiger3970,

    i'm glad, that you regained the access to the host.

    There is no rule for the physical ports. The only rule is, that you know which port is connected to which network ;)

    kind regards
    hodo
     
  18. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Ok, photo attached, as not sure of public WAN IP whilst ISP’s modem is connected to Proxmox server with no firewall. (can’t SSH in from mobile app Terminus, to copy output).
     

    Attached Files:

    #18 eiger3970, Jan 10, 2019
    Last edited: Jan 10, 2019
  19. hodo

    hodo New Member

    Joined:
    Jun 22, 2016
    Messages:
    28
    Likes Received:
    4
    Okay looks like expected in my previous post.

    Back to list of configuration steps:

    3. check your pfsense virtual hardware config -> 1 vNIC on vmbr1 (WAN), 1 vNIC on vmbr0 (LAN) , note your mac addresses on these 2 interfaces
    4. start the pfsense vm, in the vm-console there is an option to assign the network interfaces (At first boot the installer will ask you about these)
    5. set WAN interface to the vtnet adapter with the corresponding macaddress of the vNIC on vmbr1
    6. set LAN interface to the vtnet adapter with the corresponding macaddress of the vNIC on vmbr0
    7. in the vm-console give the LAN interface the static ip setting of 192.168.1.1, netmask 255.255.255.0
    8. check webaccess to https://192.168.1.1

    May i ask, why you have an tap interface? Do you use some kind of vpn?

    Can you share your configuration of vmbr2 in proxmox gui or /etc/network/interfaces?

    kind regards
    hodo
     
  20. eiger3970

    eiger3970 Member

    Joined:
    Sep 9, 2012
    Messages:
    231
    Likes Received:
    1
    Alright, I’m in the pfSense GUI.
    Thanks for your help.

    It’s messier than I’d like.
    Proxmox Network GUI shows:
    eth0 Network Device
    eth1 Network Device
    vmbr0 Linux Bridge Ports/Slaves:eth0 IP address:192.168.1.140 Subnet mask:255.255.255.0 Gateway:192.168.1.170
    vmbr1 Linux Bridge Ports/Slaves:eth1.

    PfSense vm Hardware:
    Network Device (net0) virtio=16:62:51:C6:E7:DE,bridge=vmbr1
    Network Device (net1) virtio=EE:1A:D6:89:DC:97,bridge=vmbr0

    PfSense vm Console:
    WAN (wan) -> vtnet0 ->
    LAN (lan) -> vtnet1 -> v4:192.168.1.1/24


    However my preferred setup would be:
    Gateway: 192.168.1.1, not 192.168.1.170.
    eth0 -> NIC0 (WAN?)
    eth1 -> NIC1 (LAN?)
    vmbr0 -> eth0
    vmbr1 -> eth1
    vtnet0 -> vmbr0 (WAN?)
    vtnet1 -> vmbr1 (LAN?)
    If the above is possible, I’ll decide on WAN and LAN. Possible WAN at the closest to the top of the I/O shield and LAN at the lower port...easier to remember when getting behind the machine.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice