Proxmox 5 + OPNsense + Failover IP for each VM at OVH

[thorus]

Hello,

I have a bare metal server at OVH with several FO IPs. I'm trying to change my current configuration of Proxmox to the following:
- PVE admin : 1.2.3.4
- OPNsense : 4.5.6.7
- VM1 : 4.5.6.8
- VM2 : 4.5.6.9

I also want to have a private LAN for the VMs and set up a VPN for administration (basically most of the port will be closed to outside). I can"t use NAT for VM2 in any case because on app doesn't support it. I think, Proxmox in a bridged mode is suitable for my case.
I looked at the forum and other website but couldn't find something working for me. Would you be able to help me getting working config files and IPtables rules, please?

PVE network file:

# The loopback network interface
auto lo
iface lo inet loopback
iface eno1 inet static

auto vmbr0
iface vmbr0 inet static
        address 4.5.6.7
        netmask 255.255.225.0
        gateway 4.5.6.254
        #pointtopoint 4.5.6.7
        broadcast 4.5.6.255
        bridge_ports eno1
        bridge_stp off
        bridge_fd 0
        network 4.5.6.0
#        post-up route add 4.5.6.7 dev eth0
#        post-up route add default gw 4.5.6.254
#        pre-down route del 4.5.6.7 dev eth0
#        pre-down route del default gw 4.5.6.254
        #post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp

auto vmbr1
iface vmbr1 inet static
        address 192.168.9.1
        netmask 255.2255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fdd 0

auto vmbr2
iface vmbr2 inet manual
        bridge_ports dummy1
        bridge_stp off
        bridge_fd 0

On OPNsen I have :

WAN : em0 : 4.5.6.7/24
LAN : em1 : 192.168.9.254/32
OPT1 : em2 non config

And I'm not sure for the VM's network config.
Many thanks in advance!

 

[maxicom]

Network /etc/network/interfaces

auto vmbr0
iface vmbr0 inet static
address 1.2.3.4/24
gateway 5.6.7.254
bridge_ports enp9s0f0 -eth0,eno0 ect.
bridge_stp off
bridge_fd 0


auto vmbr1
iface vmbr1 inet manual
bridge_ports enp67s0f1 -eth1,eno1 ect.
bridge_stp off
bridge_fd 0

In opensense ip public remember mask /32 ,MAC correct and vmbr0, private vmbr1 , in VM set vmbr1

 

[thorus]

Hi @maxicom, thank you very much for you quick reply.
I have followed your instructions using eno1. For vmbr1 I set bridge_ports LAN.

From the VM FW's shell I am able to ping google.com and VMx private' IPs (LAN).

For the Firewall : vmbr0 is set with the virtual MAC from OVH (02:00:00:A:B:C), vmbr1 random MAC
I want VM1 to have both public and private IPs. Should I set vmbr0 with 02:00:00:A:B:C Virtual MAC and vmbr1 to random MAC or should I just have a vmbr1 network interface with 02:00:00:A:B:C?

At the moment, I can't reach Host and VMs public IP from my desktop. Am I missing something?
How will I handle Firewall rules on private IP and their equivalent to the public IP?

When I was using one single IP, I set a 10.x.x.x WAN between PVE and VM FW. I set up OpenVPN and I was able to manage my firewall and VM with this secured VPN. How could I transpose this to my new configuration?

Kind regards,

 

[maxicom]

VM1 behind NAT

Internet <--> vmbr0-(ovh MAC-02:00:00)---OPNSENSE-vmbr1(randomMAC)<-->RandomMAC-vmbr1-(eth0VM1)

VM1 LAN and WAN same time

Internet <--> vmbr0-(ovh MAC-02:00:00)---OPNSENSE-vmbr1(randomMAC)<-->RandomMAC-vmbr1- (LANeth0-VM1-WANeth1)<-->vmbr0-(ovh-MAC-02:00:01 another MAC new public IP)

 

[thorus]

Thanks,
Yet, I thought OVH forces use to use the same virtual Mac (all my FO IP are allocated in the same virtual Mac) except the main one :

• Main IP 1.2.3.4 (not in the virtual Mac)
• ovh-MAC-02:00:00
  • FW OPNsense FO IP : 4.5.6.7
  • VM1 FO IP : 4.5.6.8
  • VM2 FO IP : 4.5.6.9

For the VM1 Lan and Wan example above you proposed to set vmbr0-(ovh-MAC-02:00:01 another MAC new public IP) then my understanding is that :
VM1 contains 2 network interfaces

• vmbr0 : OVH mac 02:00:01, shouldn't it be 02:00:00 in order to get the FO IP? then I need to manually set up the address in the /etc/netplan or etc/network/interface of the guest (VM1) to force 4.5.6.8 ?
• vmbr1 : random Mac for LAN (already working with my current config)

Cheers,

 

[maxicom]

"vmbr0 : OVH mac 02:00:01, shouldn't it be 02:00:00 in order to get the FO IP?"

yes right, you must buy new public ip FO, generate new MAC 02:00:00:xx:xx:xx and add IP to VM in netplan