I can access to an application installed in k8s inside proxmox NAT

Juliet · Jul 24, 2024

HI.

i have an application running inside a clusert k8s (nested vms inside proxmox) i port-forward to the application service on port 443:

Code:

kubectl port-forward svc/argo-cd-argocd-server 30900:443 -n argocd

when i run curl 127.0.0.1:30900 i can see the app running :

so now i want to access the web ui from the browser..i put the : http://ip_proxmox_host:30900..but it doesnt work.

i have added the port on the nat :

can you help please ?

thanks

_gabriel · Jul 24, 2024

missing -s 'x.x.x.x/24' in PREROUTING rules.

Juliet · Jul 25, 2024

_gabriel said:
missing -s 'x.x.x.x/24' in PREROUTING rules.

where exactely please ?
what is the x.x.x.x/24 IP, proxmox host ip ?

_gabriel · Jul 25, 2024

same as your MASQUERADE rule.

do not add blank lines between "post-up/down", keeping one block is recommended.

+ add this recommended rule to fix some conntrack issue when firewall is enabled :

post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

Juliet · Jul 25, 2024

_gabriel said:
same as your MASQUERADE rule.

do not add blank lines between "post-up/down", keeping one block is recommended.

+ add this recommended rule to fix some conntrack issue when firewall is enabled :
post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

i did what you suggest :

then

Code:

ifup vmbr2

but nothin happens. i'm still cant reach the UI.

_gabriel · Jul 25, 2024

fwbr+ rule was an extra.
you need to fix your PREROUTING DNAT rules, where missing the -s argument
don't forget to suppress blank lines between post- lines.
please skip screenshot and paste content into [CODE] tag

Juliet · Jul 25, 2024

hi gabriel,

i will post the code and show me please where i can add the -s :

Code:

auto vmbr2
auto vmbr2
#private sub network
iface vmbr2 inet static
        address  192.168.1.1
        netmask  255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '192.168.1.0/24' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '192.168.1.0/24' -o vmbr0 -j MASQUERADE

        post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
        post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
# redirection to the web server
post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 32768 -j DNAT --to 192.168.1.2:22
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 32768 -j DNAT --to 192.168.1.2:22


        post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 30800 -j DNAT --to 192.168.1.2:30800
        post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 30800 -j DNAT --to 192.168.1.2:30800
        post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 30900 -j DNAT --to 192.168.1.2:30900
        post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 30900 -j DNAT --to 192.168.1.2:30900

thanks a lot for your help

_gabriel · Jul 25, 2024

I can't post the answer, because you need to understand things.
please read again, "-s" present in POSTROUTING rule need to be present in PREROUTING DNAT rules too.

EDIT: ifreload -a is recommended instead ifup

Juliet · Jul 25, 2024

thanks for your help , i'm trying to understand.
i tried :

Code:

ifup vmbr2
warning: vmbr2: post-up cmd 'iptables -t nat -A PREROUTING -s vmbr0 -p tcp --dport 30900 -j DNAT --to 192.168.1.2:30900' failed: returned 2 (iptables v1.8.9 (legacy): host/network `vmbr0' not found

but it gives the error above.

_gabriel · Jul 25, 2024

look at your POSTROUTING rule, then copy/paste -s '....../24' to your PREROUTING rules.

Juliet · Jul 25, 2024

_gabriel said:
look at your POSTROUTING rule, then copy/paste -s '....../24' to your PREROUTING rules.

just for your info, i have deployed nginx..i put the same rules and it works.

Code:

post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 30800 -j DNAT --to 192.168.1.2:30800
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 30800 -j DNAT --to 192.168.1.2:30800

Juliet · Jul 25, 2024

Juliet said:
just for your info, i have deployed nginx..i put the same rules and it works.

Code:

post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 30800 -j DNAT --to 192.168.1.2:30800 post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 30800 -j DNAT --to 192.168.1.2:30800

View attachment 71826

i dont know why with

Code:

post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 30900 -j DNAT --to 192.168.1.2:30900
        post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 30900 -j DNAT --to 192.168.1.2:30900

it doesnt work

Juliet · Jul 25, 2024

Code:

warning: vmbr2: post-up cmd 'iptables -t nat -A PREROUTING -i -s '192.168.1.0/24'  vmbr0 -p tcp --dport 30900 -j DNAT --to 192.168.1.2:30900' failed: returned 2 (Bad argument `192.168.1.0/24'
Try `iptables -h' or 'iptables --help' for more information.
)

it gives this error

_gabriel · Jul 25, 2024

Juliet said:
iptables -t nat -A PREROUTING -i -s '192.168.1.0/24' vmbr0 -p tcp --dport 30900 -j DNAT --to 192.168.1.2:30900

interface need to be after -i

Juliet · Jul 25, 2024

_gabriel said:
interface need to be after -i

this time it gives no error ..but still doesnt work, i mean i dont see the web ui. i dont know why nginx work and argo-cd no.

Code:

post-up iptables -t nat -A PREROUTING -s '192.168.1.0/24' -i  vmbr0 -p tcp --dport 30900 -j DNAT --to 192.168.1.2:30900
        post-down iptables -t nat -D PREROUTING -s '192.168.1.0/24' -i  vmbr0 -p tcp --dport 30900 -j DNAT --to 192.168.1.2:30900

_gabriel · Jul 25, 2024

Désolé! I was wrong, -s seems isn't mandatory. I've not set in my rules.
Reboot host because current iptables rules is surely mixed up.
post iptables-save after host reboot.

Juliet · Jul 25, 2024

mabe i know whet is the issue..i mean why nginx work ..and argocd dont.

for nginx it is exposed as a nodeport :

Code:

nginx        NodePort    10.100.254.244   <none>        80:30800/TCP   25d

that's mean the 30800 is already exist in the worker nodes. and when i put the rules below it works :

Code:

post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 30800 -j DNAT --to 192.168.1.2:30800
             post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 30800 -j DNAT --to 192.168.1.2:30800

for argocd it has not a node port, so i have to forward the trafic like this :

Code:

kubectl port-forward svc/argo-cd-argocd-server 30900:443 -n argocd

Code:

Forwarding from 127.0.0.1:30900 -> 8080
Forwarding from [::1]:30900 -> 8080

but the port maybe does not exist in the worker node (the ufw is disables).

i dont know how to fix the issue , i have no idea

_gabriel · Jul 25, 2024

idk k8s, but it seems 30900 is only internal, and is exposed to external 443, --to 192.168.1.2:443 should work.

Juliet · Jul 25, 2024

192.168.1.2 it is not a public ip. i need to access the app using the @ip of proxmox host :

according to the doc To access the Web UI we have to port-forward to the argocd-server service on port 443

inside the vm when i put

Code:

 curl 127.0.0.1:30900

it work i can see the response..the question now how i access from the outside ?

_gabriel · Jul 25, 2024

idk Kubernetes world, seems there is another sub network.
but why it display 30900 -> 8080 ? if it's true, then you need iptables nat forward --to 192.168.1.2:8080 ...

I can access to an application installed in k8s inside proxmox NAT

Member

Attachments

Renowned Member

Member

Renowned Member

Member

Renowned Member

Member

Renowned Member

Member

Renowned Member

Member

Member

Member

Renowned Member

Member

Renowned Member

Member

Renowned Member

Member

Renowned Member

We value your privacy