[SOLVED] vncwebsocket 502 by proxy

a_running_cat

New Member
Aug 30, 2024
8
0
1
PVE 8.2.4 , nginx 1.26.0, apache2 2.4.62
  1. I create a CT with IP 10.0.0.103/24, and I install nginx for proxy
  2. Cp noVnc 1.4.0 to nginx path /mnt/dist/static/novnc
  3. Visit https://10.0.0.103/staic/novnc/vnc.html
  4. Post vncproxy Api and get port and ticket
  5. Get encodeURIComponent(ticket)
  6. try connect with url path api2/json/nodes/www/qemu/101/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D6B832%3A%3Aiu4lWyBt09qsy6p2ppHBjPjOUGbGb2Xnog%2BNg9v1tPk7bMcrkqR41P3RIMEZu2xGsEEzs%2F05v74sAbGZq%2BjpWtLTfl341%2BRIEJtu8FTEhH015MY2kPpi73rr7RBZRr4Fbpczc%2FrtgmdX%2BaoBk88%2BTLl%2By0i3x11BRcLPDoyBSyCJZGtHpwj6hdsNdwmoMUhqNgAaOhF76KkBXyGY5msm7%2Bgv5wr7jxXZWK5Kp7paGs5UN%2B2KUK8aIAPWaOSo1WedledB30ceZdrVVF3tvSA3uMsDQWquxWPChAXLKYh3VHKsuKoJBqJ6oWJ0BwekJnJgtLgEauBJsyUky6BjPVubYQ%3D%3D
  7. Then I get 502 ...
  8. I also try replace nginx with apache2 and get the same error
  9. By the way, sometime I can connect lxc vncwebsocket success, but most time I only get 502.
1.PNG

Here is my nginx conf:
NGINX:
server{
    listen 0.0.0.0:443 ssl;
    server_name www.mytest.com;
    ssl_certificate      certs/server.crt;
    ssl_certificate_key  certs/server.key;

    ssl_session_cache    shared:SSL:1m;
    ssl_session_timeout  5m;

    ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers  on;

    location /api2 {
        client_max_body_size 1000m;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Forwarded-Ssl on;
        proxy_set_header Cookie "PVEAuthCookie=PVE%3Aroot%40pam%3A66D6CB4B%3A%3AozA6bNAfd%2Fg%2B2GkstkeYHnmHExFJ9hPIaFKN6tY49jKj2H3F%2Fi3CQUoKYClryg0%2BmESMW93akKh7zW1MewOYbE3ZbTj%2FWBwNiuirKKVYvBOTnTrf5WDsoAP2CYlFD5wwZf4HICOgnx91rCCp%2FNRMR9nRhxbHLJy2BGvSgTEh6gZB%2FfJ%2FhFQeUWE9Pi%2FLGH0E0SZDng356iYH7FJY7j5duipUgz35cTf5%2BLxRHHrqN4IeRiJBIScY%2B7v4xclbW41xaHRm6wrmIS37WJPzkFKyNhk896rErI5rtzxyw4kbff6em1QqubobVYSDXx0n7YqTP0hLg86dq51E2Svv2v%2FFXw%3D%3D";

        proxy_connect_timeout  3600s;
        proxy_read_timeout  3600s;
        proxy_send_timeout  3600s;
        send_timeout  3600s;
        proxy_ssl_verify off;
        proxy_pass https://10.0.0.1:8006;
    }
location /static {
        alias /mnt/dist/static/;
    }

    location @router {
                rewrite ^.*$ /index.html last;
    }
}

Code:
Nginx Error:
2024/09/03 08:51:33 [error] 40#40: *3 upstream prematurely closed connection while reading response header from upstream, client: 10.0.0.103, server: www.industrial-sandbox.com, request: "GET /api2/json/nodes/www/qemu/101/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D6CDF9%3A%3AeG0ibCM%2BSre3nKqEPtjSUOs%2FDn0PUpmVZhoEiCwaPofFqtLbjZnetpMbBs0GvXOjdbcB7x9BChoJMpnw1I2kl87%2FFXOKRaPAVVKfMf0cYlsvLZ6U90J2yAzg2nOw%2F1VH2WZEFKU%2BStl0kdNubcmJXL651qaVik5ZlVBFgO5zNPUnbxvYWasnG%2FLoSXbV3eg9xKLIKMrvkTLx92IuJUwV93ZXcyT6Ahsg5hahWvtd4dOU0brM5PO0W5l64QgJsa%2F7GcdnY%2FVG35nmp5m2LEl1ey4tKKkLmaM4%2BNMcCN%2BcE4BY1aoCKT7QOucGyT6AK9FHazGsRzrbkg3dFnrZjdAj%2Fg%3D%3D HTTP/1.1", upstream: "https://10.0.0.1:8006/api2/json/nodes/www/qemu/101/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D6CDF9%3A%3AeG0ibCM%2BSre3nKqEPtjSUOs%2FDn0PUpmVZhoEiCwaPofFqtLbjZnetpMbBs0GvXOjdbcB7x9BChoJMpnw1I2kl87%2FFXOKRaPAVVKfMf0cYlsvLZ6U90J2yAzg2nOw%2F1VH2WZEFKU%2BStl0kdNubcmJXL651qaVik5ZlVBFgO5zNPUnbxvYWasnG%2FLoSXbV3eg9xKLIKMrvkTLx92IuJUwV93ZXcyT6Ahsg5hahWvtd4dOU0brM5PO0W5l64QgJsa%2F7GcdnY%2FVG35nmp5m2LEl1ey4tKKkLmaM4%2BNMcCN%2BcE4BY1aoCKT7QOucGyT6AK9FHazGsRzrbkg3dFnrZjdAj%2Fg%3D%3D", host: "10.0.0.103"
10.0.0.103 - - [03/Sep/2024:08:51:33 +0000] "GET /api2/json/nodes/www/qemu/101/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D6CDF9%3A%3AeG0ibCM%2BSre3nKqEPtjSUOs%2FDn0PUpmVZhoEiCwaPofFqtLbjZnetpMbBs0GvXOjdbcB7x9BChoJMpnw1I2kl87%2FFXOKRaPAVVKfMf0cYlsvLZ6U90J2yAzg2nOw%2F1VH2WZEFKU%2BStl0kdNubcmJXL651qaVik5ZlVBFgO5zNPUnbxvYWasnG%2FLoSXbV3eg9xKLIKMrvkTLx92IuJUwV93ZXcyT6Ahsg5hahWvtd4dOU0brM5PO0W5l64QgJsa%2F7GcdnY%2FVG35nmp5m2LEl1ey4tKKkLmaM4%2BNMcCN%2BcE4BY1aoCKT7QOucGyT6AK9FHazGsRzrbkg3dFnrZjdAj%2Fg%3D%3D HTTP/1.1" 502 150 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0" "-"

Code:
Apache Error:
[Tue Sep 03 02:56:30.933341 2024] [proxy_http:error] [pid 42:tid 42] (70014)End of file found: [client 10.0.0.103:53716] AH01102: error reading status line from remote server 10.0.0.1:8006
[Tue Sep 03 02:56:30.933366 2024] [proxy:error] [pid 42:tid 42] [client 10.0.0.103:53716] AH00898: Error reading from remote server returned by /api2/json/nodes/www/lxc/113/vncwebsocket
10.0.0.103 - - [03/Sep/2024:02:56:25 +0000] "GET /api2/json/nodes/www/lxc/113/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D67AB9%3A%3AJfnYDpSYJGZmaj2YGcWlYqtc05rFgic6yRVNqFwntWtLfT9v9NaSyWW%2F9x2T837QrbPokXkeuD9%2FhtjA1U4Qmc2oIWAlaDjJvQPeiriL%2BJ7WtD6x5bWhnb0xfREBOFMsaKAP596SVJehsgTiGRVxl%2BTzvVPR9uKCtGrIpuUtnpJ9gAIa3JDYLsibY13NmZ5VSbzkYzVZ7eYCutSKRNJ%2FrSG2Gette0fWyBpVFjtuKglYpnSSeU3Nmn4FLXEb86RCWdxRYDUGfjKAdZ0Rjj%2BifNLfTV%2BMJjPMk2xG1%2Fd0p3nQKL8tw4LsAXLBL2h3%2BaxZmYSw8AVCmyj%2BdfL9JTpDuA%3D%3D HTTP/1.1" 502 341
10.0.0.103 - - [03/Sep/2024:03:18:15 +0000] "GET /api2/json/nodes/www/lxc/113/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D67AB9%3A%3AJfnYDpSYJGZmaj2YGcWlYqtc05rFgic6yRVNqFwntWtLfT9v9NaSyWW%2F9x2T837QrbPokXkeuD9%2FhtjA1U4Qmc2oIWAlaDjJvQPeiriL%2BJ7WtD6x5bWhnb0xfREBOFMsaKAP596SVJehsgTiGRVxl%2BTzvVPR9uKCtGrIpuUtnpJ9gAIa3JDYLsibY13NmZ5VSbzkYzVZ7eYCutSKRNJ%2FrSG2Gette0fWyBpVFjtuKglYpnSSeU3Nmn4FLXEb86RCWdxRYDUGfjKAdZ0Rjj%2BifNLfTV%2BMJjPMk2xG1%2Fd0p3nQKL8tw4LsAXLBL2h3%2BaxZmYSw8AVCmyj%2BdfL9JTpDuA%3D%3D HTTP/1.1" 401 13
10.0.0.103 - - [03/Sep/2024:03:19:10 +0000] "GET /api2/json/nodes/www/lxc/113/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D68008%3A%3AfvGqCfHj8oqjUjdIdkVPYCAAyJH5xyi9qGxNWe9%2BtnP30FQl%2BOByAHCSpE4tBigUuDPLIeFrFBV0Q6UvFQKd%2BxPyEdeEtAj43pMj1NGVOJJmn0hmncW%2FkKiuEGdWbi3iS3M69wOQ07wbjoaxyisqTJZT78nBybe0xFM8w38j8ncNN02hJKB%2B9FkfUdPan0Mdp7sJqse7aVnlQfZ7yb7qq13Gsj9G93rQcGaWuk%2BNqSu5le1ZlOckoyxivdHn6xF6hqJsPqW0vLHQrS1cJtfS9fbd8idmY%2BA4CU%2BynTD%2ByLKeNDuBeDndgOv2Q8cxI%2BUJz4%2BKHRxzeDQ63xU5zE%2B1ww%3D%3D HTTP/1.1" 101 -
[Tue Sep 03 03:21:45.282835 2024] [proxy_http:error] [pid 39:tid 39] (70014)End of file found: [client 10.0.0.103:53442] AH01102: error reading status line from remote server 10.0.0.1:8006
[Tue Sep 03 03:21:45.282859 2024] [proxy:error] [pid 39:tid 39] [client 10.0.0.103:53442] AH00898: Error reading from remote server returned by /api2/json/nodes/www/qemu/101/vncwebsocket
10.0.0.103 - - [03/Sep/2024:03:21:40 +0000] "GET /api2/json/nodes/www/qemu/101/vncwebsocket?port=5901&vncticket=PVEVNC%3A66D680A2%3A%3AbbmHwo5wF7yPtheqYHupnIXip9KmhO428MsTdABqPcD9g79t4qHKoFlv%2BbAc0fANQ5Nz4CFIrbzVJwO1lkMhUpnaRSVMDDpjAXy81uWWdxpC8DswOP6yIjypyMu1r6hiOKG%2Fb%2Bd5vZ4TuJWgN%2BKxiC%2Ffjmv5MHrIJUX%2FpdT14i4%2B7q8rYHHVTpyeCC3DyWazzMErb5d1J2geiSWc%2B%2Bq3A%2FqU0NZvNjUvkYaObdFQKt%2F4Xe5izA90rUIQYRwsZcirmtdstLUx71bPZ5woAd7gDRqS3O%2Fu7%2F9i56DjeLK3HEnXVqgzZzp8Fq1EKkYVvwSGA%2FyxXqCdNRGX3ouZqhGbbw%3D%3D HTTP/1.1" 502 341
[Tue Sep 03 03:22:48.312491 2024] [proxy_http:error] [pid 41:tid 41] (70014)End of file found: [client 10.0.0.103:35760] AH01102: error reading status line from remote server 10.0.0.1:8006
[Tue Sep 03 03:22:48.312519 2024] [proxy:error] [pid 41:tid 41] [client 10.0.0.103:35760] AH00898: Error reading from remote server returned by /api2/json/nodes/www/qemu/101/vncwebsocket
 
Last edited:
New situation
websocket worked after I restart pveproxy service, but no matter which ID (qemu or lxc) I use to connect, the VNC result I get is the same.
P1: I connect to lxc 136
lxc连接2.PNG
P2: I connect to qemu 101
使用qemu连接但是却连上了ct.PNG

And The VNC I received is lxc 113!

I thought it was caused by operating in this lxc(ID 113, IP 10.0.0.103), so I switched to a physical machine (172.0.16.xxx), but the result was the same.

I'm a little dizzy, what should I do next?
 
PVE 8.2.4 , nginx 1.26.0, apache2 2.4.62
  1. I create a CT with IP 10.0.0.103/24, and I install nginx for proxy
  2. Cp noVnc 1.4.0 to nginx path /mnt/dist/static/novnc
  3. Visit https://10.0.0.103/staic/novnc/vnc.html
  4. Post vncproxy Api and get port and ticket
  5. Get encodeURIComponent(ticket)
  6. try connect with url path api2/json/nodes/www/qemu/101/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D6B832%3A%3Aiu4lWyBt09qsy6p2ppHBjPjOUGbGb2Xnog%2BNg9v1tPk7bMcrkqR41P3RIMEZu2xGsEEzs%2F05v74sAbGZq%2BjpWtLTfl341%2BRIEJtu8FTEhH015MY2kPpi73rr7RBZRr4Fbpczc%2FrtgmdX%2BaoBk88%2BTLl%2By0i3x11BRcLPDoyBSyCJZGtHpwj6hdsNdwmoMUhqNgAaOhF76KkBXyGY5msm7%2Bgv5wr7jxXZWK5Kp7paGs5UN%2B2KUK8aIAPWaOSo1WedledB30ceZdrVVF3tvSA3uMsDQWquxWPChAXLKYh3VHKsuKoJBqJ6oWJ0BwekJnJgtLgEauBJsyUky6BjPVubYQ%3D%3D
  7. Then I get 502 ...
  8. I also try replace nginx with apache2 and get the same error
  9. By the way, sometime I can connect lxc vncwebsocket success, but most time I only get 502.
View attachment 74129

Here is my nginx conf:
NGINX:
server{
    listen 0.0.0.0:443 ssl;
    server_name www.mytest.com;
    ssl_certificate      certs/server.crt;
    ssl_certificate_key  certs/server.key;

    ssl_session_cache    shared:SSL:1m;
    ssl_session_timeout  5m;

    ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers  on;

    location /api2 {
        client_max_body_size 1000m;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Forwarded-Ssl on;
        proxy_set_header Cookie "PVEAuthCookie=PVE%3Aroot%40pam%3A66D6CB4B%3A%3AozA6bNAfd%2Fg%2B2GkstkeYHnmHExFJ9hPIaFKN6tY49jKj2H3F%2Fi3CQUoKYClryg0%2BmESMW93akKh7zW1MewOYbE3ZbTj%2FWBwNiuirKKVYvBOTnTrf5WDsoAP2CYlFD5wwZf4HICOgnx91rCCp%2FNRMR9nRhxbHLJy2BGvSgTEh6gZB%2FfJ%2FhFQeUWE9Pi%2FLGH0E0SZDng356iYH7FJY7j5duipUgz35cTf5%2BLxRHHrqN4IeRiJBIScY%2B7v4xclbW41xaHRm6wrmIS37WJPzkFKyNhk896rErI5rtzxyw4kbff6em1QqubobVYSDXx0n7YqTP0hLg86dq51E2Svv2v%2FFXw%3D%3D";

        proxy_connect_timeout  3600s;
        proxy_read_timeout  3600s;
        proxy_send_timeout  3600s;
        send_timeout  3600s;
        proxy_ssl_verify off;
        proxy_pass https://10.0.0.1:8006;
    }
location /static {
        alias /mnt/dist/static/;
    }

    location @router {
                rewrite ^.*$ /index.html last;
    }
}

Code:
Nginx Error:
2024/09/03 08:51:33 [error] 40#40: *3 upstream prematurely closed connection while reading response header from upstream, client: 10.0.0.103, server: www.industrial-sandbox.com, request: "GET /api2/json/nodes/www/qemu/101/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D6CDF9%3A%3AeG0ibCM%2BSre3nKqEPtjSUOs%2FDn0PUpmVZhoEiCwaPofFqtLbjZnetpMbBs0GvXOjdbcB7x9BChoJMpnw1I2kl87%2FFXOKRaPAVVKfMf0cYlsvLZ6U90J2yAzg2nOw%2F1VH2WZEFKU%2BStl0kdNubcmJXL651qaVik5ZlVBFgO5zNPUnbxvYWasnG%2FLoSXbV3eg9xKLIKMrvkTLx92IuJUwV93ZXcyT6Ahsg5hahWvtd4dOU0brM5PO0W5l64QgJsa%2F7GcdnY%2FVG35nmp5m2LEl1ey4tKKkLmaM4%2BNMcCN%2BcE4BY1aoCKT7QOucGyT6AK9FHazGsRzrbkg3dFnrZjdAj%2Fg%3D%3D HTTP/1.1", upstream: "https://10.0.0.1:8006/api2/json/nodes/www/qemu/101/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D6CDF9%3A%3AeG0ibCM%2BSre3nKqEPtjSUOs%2FDn0PUpmVZhoEiCwaPofFqtLbjZnetpMbBs0GvXOjdbcB7x9BChoJMpnw1I2kl87%2FFXOKRaPAVVKfMf0cYlsvLZ6U90J2yAzg2nOw%2F1VH2WZEFKU%2BStl0kdNubcmJXL651qaVik5ZlVBFgO5zNPUnbxvYWasnG%2FLoSXbV3eg9xKLIKMrvkTLx92IuJUwV93ZXcyT6Ahsg5hahWvtd4dOU0brM5PO0W5l64QgJsa%2F7GcdnY%2FVG35nmp5m2LEl1ey4tKKkLmaM4%2BNMcCN%2BcE4BY1aoCKT7QOucGyT6AK9FHazGsRzrbkg3dFnrZjdAj%2Fg%3D%3D", host: "10.0.0.103"
10.0.0.103 - - [03/Sep/2024:08:51:33 +0000] "GET /api2/json/nodes/www/qemu/101/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D6CDF9%3A%3AeG0ibCM%2BSre3nKqEPtjSUOs%2FDn0PUpmVZhoEiCwaPofFqtLbjZnetpMbBs0GvXOjdbcB7x9BChoJMpnw1I2kl87%2FFXOKRaPAVVKfMf0cYlsvLZ6U90J2yAzg2nOw%2F1VH2WZEFKU%2BStl0kdNubcmJXL651qaVik5ZlVBFgO5zNPUnbxvYWasnG%2FLoSXbV3eg9xKLIKMrvkTLx92IuJUwV93ZXcyT6Ahsg5hahWvtd4dOU0brM5PO0W5l64QgJsa%2F7GcdnY%2FVG35nmp5m2LEl1ey4tKKkLmaM4%2BNMcCN%2BcE4BY1aoCKT7QOucGyT6AK9FHazGsRzrbkg3dFnrZjdAj%2Fg%3D%3D HTTP/1.1" 502 150 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0" "-"

Code:
Apache Error:
[Tue Sep 03 02:56:30.933341 2024] [proxy_http:error] [pid 42:tid 42] (70014)End of file found: [client 10.0.0.103:53716] AH01102: error reading status line from remote server 10.0.0.1:8006
[Tue Sep 03 02:56:30.933366 2024] [proxy:error] [pid 42:tid 42] [client 10.0.0.103:53716] AH00898: Error reading from remote server returned by /api2/json/nodes/www/lxc/113/vncwebsocket
10.0.0.103 - - [03/Sep/2024:02:56:25 +0000] "GET /api2/json/nodes/www/lxc/113/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D67AB9%3A%3AJfnYDpSYJGZmaj2YGcWlYqtc05rFgic6yRVNqFwntWtLfT9v9NaSyWW%2F9x2T837QrbPokXkeuD9%2FhtjA1U4Qmc2oIWAlaDjJvQPeiriL%2BJ7WtD6x5bWhnb0xfREBOFMsaKAP596SVJehsgTiGRVxl%2BTzvVPR9uKCtGrIpuUtnpJ9gAIa3JDYLsibY13NmZ5VSbzkYzVZ7eYCutSKRNJ%2FrSG2Gette0fWyBpVFjtuKglYpnSSeU3Nmn4FLXEb86RCWdxRYDUGfjKAdZ0Rjj%2BifNLfTV%2BMJjPMk2xG1%2Fd0p3nQKL8tw4LsAXLBL2h3%2BaxZmYSw8AVCmyj%2BdfL9JTpDuA%3D%3D HTTP/1.1" 502 341
10.0.0.103 - - [03/Sep/2024:03:18:15 +0000] "GET /api2/json/nodes/www/lxc/113/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D67AB9%3A%3AJfnYDpSYJGZmaj2YGcWlYqtc05rFgic6yRVNqFwntWtLfT9v9NaSyWW%2F9x2T837QrbPokXkeuD9%2FhtjA1U4Qmc2oIWAlaDjJvQPeiriL%2BJ7WtD6x5bWhnb0xfREBOFMsaKAP596SVJehsgTiGRVxl%2BTzvVPR9uKCtGrIpuUtnpJ9gAIa3JDYLsibY13NmZ5VSbzkYzVZ7eYCutSKRNJ%2FrSG2Gette0fWyBpVFjtuKglYpnSSeU3Nmn4FLXEb86RCWdxRYDUGfjKAdZ0Rjj%2BifNLfTV%2BMJjPMk2xG1%2Fd0p3nQKL8tw4LsAXLBL2h3%2BaxZmYSw8AVCmyj%2BdfL9JTpDuA%3D%3D HTTP/1.1" 401 13
10.0.0.103 - - [03/Sep/2024:03:19:10 +0000] "GET /api2/json/nodes/www/lxc/113/vncwebsocket?port=5902&vncticket=PVEVNC%3A66D68008%3A%3AfvGqCfHj8oqjUjdIdkVPYCAAyJH5xyi9qGxNWe9%2BtnP30FQl%2BOByAHCSpE4tBigUuDPLIeFrFBV0Q6UvFQKd%2BxPyEdeEtAj43pMj1NGVOJJmn0hmncW%2FkKiuEGdWbi3iS3M69wOQ07wbjoaxyisqTJZT78nBybe0xFM8w38j8ncNN02hJKB%2B9FkfUdPan0Mdp7sJqse7aVnlQfZ7yb7qq13Gsj9G93rQcGaWuk%2BNqSu5le1ZlOckoyxivdHn6xF6hqJsPqW0vLHQrS1cJtfS9fbd8idmY%2BA4CU%2BynTD%2ByLKeNDuBeDndgOv2Q8cxI%2BUJz4%2BKHRxzeDQ63xU5zE%2B1ww%3D%3D HTTP/1.1" 101 -
[Tue Sep 03 03:21:45.282835 2024] [proxy_http:error] [pid 39:tid 39] (70014)End of file found: [client 10.0.0.103:53442] AH01102: error reading status line from remote server 10.0.0.1:8006
[Tue Sep 03 03:21:45.282859 2024] [proxy:error] [pid 39:tid 39] [client 10.0.0.103:53442] AH00898: Error reading from remote server returned by /api2/json/nodes/www/qemu/101/vncwebsocket
10.0.0.103 - - [03/Sep/2024:03:21:40 +0000] "GET /api2/json/nodes/www/qemu/101/vncwebsocket?port=5901&vncticket=PVEVNC%3A66D680A2%3A%3AbbmHwo5wF7yPtheqYHupnIXip9KmhO428MsTdABqPcD9g79t4qHKoFlv%2BbAc0fANQ5Nz4CFIrbzVJwO1lkMhUpnaRSVMDDpjAXy81uWWdxpC8DswOP6yIjypyMu1r6hiOKG%2Fb%2Bd5vZ4TuJWgN%2BKxiC%2Ffjmv5MHrIJUX%2FpdT14i4%2B7q8rYHHVTpyeCC3DyWazzMErb5d1J2geiSWc%2B%2Bq3A%2FqU0NZvNjUvkYaObdFQKt%2F4Xe5izA90rUIQYRwsZcirmtdstLUx71bPZ5woAd7gDRqS3O%2Fu7%2F9i56DjeLK3HEnXVqgzZzp8Fq1EKkYVvwSGA%2FyxXqCdNRGX3ouZqhGbbw%3D%3D HTTP/1.1" 502 341
[Tue Sep 03 03:22:48.312491 2024] [proxy_http:error] [pid 41:tid 41] (70014)End of file found: [client 10.0.0.103:35760] AH01102: error reading status line from remote server 10.0.0.1:8006
[Tue Sep 03 03:22:48.312519 2024] [proxy:error] [pid 41:tid 41] [client 10.0.0.103:35760] AH00898: Error reading from remote server returned by /api2/json/nodes/www/qemu/101/vncwebsocket
I get more message:
● pveproxy.service - PVE API Proxy Server
Active: active (running) since Wed 2024-09-04 10:29:41 CST; 5h 32min ago
Process: 1553633 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
Process: 1553655 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
Main PID: 1553869 (pveproxy)
Tasks: 8 (limit: 76416)
Memory: 283.2M
CPU: 5min 38.800s
CGroup: /system.slice/pveproxy.service
├─ 833085 "pveproxy worker (shutdown)"
├─ 855967 "pveproxy worker"
├─ 867301 "pveproxy worker"
├─ 921543 "pveproxy worker"
├─1553869 pveproxy
├─1637842 "pveproxy worker (shutdown)"
├─1692911 "pveproxy worker (shutdown)"
└─3640016 "pveproxy worker (shutdown)"

Sep 04 15:56:14 www pveproxy[1553869]: worker 867301 started
Sep 04 15:56:15 www pveproxy[867300]: worker exit
Sep 04 15:57:57 www pveproxy[855964]: worker exit
Sep 04 15:58:44 www pveproxy[667960]: worker exit
Sep 04 15:58:45 www pveproxy[625482]: worker exit
Sep 04 16:01:09 www pveproxy[833090]: connect to 'localhost:5902' failed: Connection refused at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 558.
Sep 04 16:01:14 www pveproxy[833090]: worker exit
Sep 04 16:01:14 www pveproxy[1553869]: worker 833090 finished


Code:
557         tcp_connect $remhost, $remport, sub {
 558             my ($fh) = @_
 559                 or die "connect to '$remhost:$remport' failed: $!";
 560
 561             $self->dprint("CONNECTed to '$remhost:$remport'");
 562
 563             $reqstate->{proxyhdl} = AnyEvent::Handle->new(
 564                 fh => $fh,
 565                 rbuf_max => $max_payload_size,
 566                 wbuf_max => $max_payload_size*5,
 567                 timeout => 5,
 568                 on_eof => sub {
 569                     my ($hdl) = @_;
 570                     eval {
 571                         $self->log_aborted_request($reqstate);
 572                         $self->client_do_disconnect($reqstate);
 573                     };
 574                     if (my $err = $@) { syslog('err', $err); }
 575                 },
 576                 on_error => sub {
 577                     my ($hdl, $fatal, $message) = @_;
 578                     eval {
 579                         $self->log_aborted_request($reqstate, $message);
 580                         $self->client_do_disconnect($reqstate);
 581                     };
 582                     if (my $err = $@) { syslog('err', "$err"); }
 583                 });
 584
 585             my $proxyhdlreader = sub {
 586                 my ($hdl) = @_;
 587
 588                 my $len = length($hdl->{rbuf});
 589                 my $data = substr($hdl->{rbuf}, 0, $len > $max_payload_size ? $max_payload_size : $len, '');
 590
 591                 my $string = $encode->(\$data);
 592
 593                 $reqstate->{hdl}->push_write($string) if $reqstate->{hdl};
 594             };

I still don't understand what went wrong at the moment.
 
My suggestions here:

1) use pve API instead of hard coded and expiring cookie
In Nginx reverse proxy server use something like...

Code:
proxy_set_header "Authorization" "PVEAPIToken=foo@pam!some-api=XXXXX-XXX-XXX-XXXX-XXXXXXX";


2) here is the (php) code below that you can use as reference for code I have which works ok. It shows how I build the end novnc URL.

PHP:
$vncCredentials // <--- This is after you make your POST request to `vncproxy` with the param of `websocket: 1`
$vncUrl = 'https://nginx-proxy.foobar.com/?autoconnect=true&host=nginx-proxy.foobar.com&port=443&encrypt=1';
$vncUrl .= '&password=' . urlencode($vncCredentials->ticket);
$vncUrl .= '&resize=remote';
$vncUrl .= '&path=' . urlencode('api2/json/nodes/' . $this->node . '/' . $this->type . '/' . $this->vmid . '/vncwebsocket?port=' . $vncCredentials->port . '&vncticket=' . urlencode($vncCredentials->ticket));

The above url formed is not the WSS url but the URL that pre populates your novnc application (in your case stored under 'static' folder) so you will need to slightly adjust the above to make it work for you with your prefix URI. You can confirm everything in the noVNC settings box that has the correct PATH, HOST and PASSWORD after pageload

Hopefully this helps iron out any weirdness.
 
Last edited:
My suggestions here:

1) use pve API instead of hard coded and expiring cookie
In Nginx reverse proxy server use something like...

Code:
proxy_set_header "Authorization" "PVEAPIToken=foo@pam!some-api=XXXXX-XXX-XXX-XXXX-XXXXXXX";


2) here is the (php) code below that you can use as reference for code I have which works ok. It shows how I build the end novnc URL.

PHP:
$vncCredentials // <--- This is after you make your POST request to `vncproxy` with the param of `websocket: 1`
$vncUrl = 'https://nginx-proxy.foobar.com/?autoconnect=true&host=nginx-proxy.foobar.com&port=443&encrypt=1';
$vncUrl .= '&password=' . urlencode($vncCredentials->ticket);
$vncUrl .= '&resize=remote';
$vncUrl .= '&path=' . urlencode('api2/json/nodes/' . $this->node . '/' . $this->type . '/' . $this->vmid . '/vncwebsocket?port=' . $vncCredentials->port . '&vncticket=' . urlencode($vncCredentials->ticket));

The above url formed is not the WSS url but the URL that pre populates your novnc application (in your case stored under 'static' folder) so you will need to slightly adjust the above to make it work for you with your prefix URI. You can confirm everything in the noVNC settings box that has the correct PATH, HOST and PASSWORD after pageload

Hopefully this helps iron out any weirdness.
I understand the reason now.
When I post vncproxy api, pve will start vnc proxy and wait to connect and timeout after 10s.
I encoded and copied/pasted the ticket, which took over 10 seconds, so I always get 502.

Code:
Sep 05 16:07:46 test pvedaemon[21778]: <root@pam!test> starting task UPID:test:000058E0:000ADA80:66D966D2:vncproxy:101:root@pam!test:
Sep 05 16:07:46 test pvedaemon[22752]: starting vnc proxy UPID:test:000058E0:000ADA80:66D966D2:vncproxy:101:root@pam!test:
Sep 05 16:07:56 test pvedaemon[22752]: connection timed out
Sep 05 16:07:56 test pvedaemon[21778]: <root@pam!test> end task UPID:test:000058E0:000ADA80:66D966D2:vncproxy:101:root@pam!test: connection timed out
 
Now I know why.
1.post vncproxy get ticket
2.pve server will start vnc proxy and listen such as 5900 and timeout after 10s.
3.if we cannot establish a connection as soon as possible, we will get 502.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!