can you ping the IP address from the pc where you try to connect by RDP? if can you telnet on the RDP port? (in other words, be sure you can reach RDP address+port)
if you mean the pve VNC console, then VNC is done by pve server, which can reach the windows server, and then redirected to your browser, it is not a direct vnc connection (you would need a vnc serve in windows for that)
Marco