All CT turned to Read only file system

fabian said:
just re-read the messages - those are not nesting related, but setting sysctls that are not settable in a container because they would affect the whole host. does the container actually not work, or are you just worried about those messages?
Click to expand...
The container starts but no network. And restarting services returns permission error I didn't have before. Example with httpd :

Code: 
# service httpd restart
/etc/init.d/functions: ligne63: /dev/stderr: Permission non accordée
Redirecting to /bin/systemctl restart httpd.service

Restarting network fails (from journalctl) :

Code: 
-- L'unité (unit) network.service a commencé à démarrer.
mars 29 10:56:27 domain.com systemd-sysctl[908]: Failed to write '0' to '/proc/sys/kernel/yama/ptrace_scope': Read-only file
mars 29 10:56:27 domain.com systemd-sysctl[908]: Failed to write '16' to '/proc/sys/kernel/sysrq': Read-only file system
mars 29 10:56:27 domain.com systemd-sysctl[908]: Failed to write '1' to '/proc/sys/kernel/core_uses_pid': Read-only file syst
mars 29 10:56:27 domain.com systemd-sysctl[908]: Failed to write '1' to '/proc/sys/kernel/kptr_restrict': Read-only file syst
mars 29 10:56:27 domain.com systemd-sysctl[908]: Failed to write '1' to '/proc/sys/fs/protected_hardlinks': Read-only file sy
mars 29 10:56:27 domain.com systemd-sysctl[908]: Failed to write '1' to '/proc/sys/fs/protected_symlinks': Read-only file sys
mars 29 10:56:27 domain.com network[880]: Bringing up loopback interface:  [  OK  ]
mars 29 10:56:27 domain.com network[880]: Bringing up interface eth0:  RTNETLINK answers: File exists
mars 29 10:56:27 domain.com network[880]: [  OK  ]
mars 29 10:56:27 domain.com network[880]: Bringing up interface venet0:  ERROR     : [/etc/sysconfig/network-scripts/ifup-eth
mars 29 10:56:27 domain.com network[880]: [FAILED]
mars 29 10:56:27 domain.com network[880]: RTNETLINK answers: File exists
mars 29 10:56:27 domain.com network[880]: RTNETLINK answers: File exists
mars 29 10:56:27 domain.com network[880]: RTNETLINK answers: File exists
mars 29 10:56:27 domain.com network[880]: RTNETLINK answers: File exists
mars 29 10:56:27 domain.com network[880]: RTNETLINK answers: File exists
mars 29 10:56:27 domain.com network[880]: RTNETLINK answers: File exists
mars 29 10:56:27 domain.com network[880]: RTNETLINK answers: File exists
mars 29 10:56:27 domain.com network[880]: RTNETLINK answers: File exists
mars 29 10:56:27 domain.com network[880]: RTNETLINK answers: File exists
mars 29 10:56:27 domain.com systemd-sysctl[1110]: Failed to write '0' to '/proc/sys/kernel/yama/ptrace_scope': Read-only file
mars 29 10:56:27 domain.com systemd-sysctl[1110]: Failed to write '16' to '/proc/sys/kernel/sysrq': Read-only file system
mars 29 10:56:27 domain.com systemd-sysctl[1110]: Failed to write '1' to '/proc/sys/kernel/core_uses_pid': Read-only file sys
mars 29 10:56:27 domain.com systemd-sysctl[1110]: Failed to write '1' to '/proc/sys/kernel/kptr_restrict': Read-only file sys
mars 29 10:56:27 domain.com systemd-sysctl[1110]: Failed to write '1' to '/proc/sys/fs/protected_hardlinks': Read-only file s
mars 29 10:56:27 domain.com systemd-sysctl[1110]: Failed to write '1' to '/proc/sys/fs/protected_symlinks': Read-only file sy
mars 29 10:56:27 domain.com systemd[1]: network.service: control process exited, code=exited status=1
mars 29 10:56:27 domain.com systemd[1]: Failed to start LSB: Bring up/down networking.
-- Subject: L'unité (unit) network.service a échoué
 
Last edited:
how did you create the container?

the network error messages would indicate that there are multiple systems attempting to bring up the network interfaces, and they mention a 'venet0' NIC that is not managed by PVE..
 
At the begining the CT was on a PVE3. The CT has been backupped and restored on this PVE6. All worked fine for the 3 CT until a few days for the 2 CentOS 7.

The error with venet0 has been fixed (maybe due to multi add/delete eth). But the main problem is how to fix all the errors "Failed to write '1' to '/proc/sys/..." ?
 
there is no problem - a container cannot write there, so you can either ignore that or make it not attempt to write there.
 
When I start the CT with debug I can see problem with cgroups. But I have no idea how to fix it.

Code: 
lxc-start 100 20220330094646.174 DEBUG    conf - conf.c:idmaptool_on_path_and_privileged:2741 - The binary "/usr/bin/newuidmap" does have the setuid bit set
lxc-start 100 20220330094646.174 DEBUG    conf - conf.c:idmaptool_on_path_and_privileged:2741 - The binary "/usr/bin/newgidmap" does have the setuid bit set
lxc-start 100 20220330094646.174 DEBUG    conf - conf.c:lxc_map_ids:2809 - Functional newuidmap and newgidmap binary found
lxc-start 100 20220330094646.182 INFO     start - start.c:do_start:1085 - Unshared CLONE_NEWNET
lxc-start 100 20220330094646.183 DEBUG    cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2874 - Set controller "memory.limit_in_bytes" set to "4294967296"
lxc-start 100 20220330094646.183 DEBUG    cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2874 - Set controller "memory.memsw.limit_in_bytes" set to "9663676416"
lxc-start 100 20220330094646.183 DEBUG    cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2874 - Set controller "cpu.cfs_period_us" set to "100000"
lxc-start 100 20220330094646.183 DEBUG    cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2874 - Set controller "cpu.cfs_quota_us" set to "100000"
lxc-start 100 20220330094646.183 DEBUG    cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2874 - Set controller "cpu.shares" set to "1024"
lxc-start 100 20220330094646.183 INFO     cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2879 - Limits for the legacy cgroup hierarchies have been setup
lxc-start 100 20220330094646.184 DEBUG    conf - conf.c:idmaptool_on_path_and_privileged:2741 - The binary "/usr/bin/newuidmap" does have the setuid bit set
lxc-start 100 20220330094646.184 DEBUG    conf - conf.c:idmaptool_on_path_and_privileged:2741 - The binary "/usr/bin/newgidmap" does have the setuid bit set
lxc-start 100 20220330094646.184 DEBUG    conf - conf.c:lxc_map_ids:2809 - Functional newuidmap and newgidmap binary found
lxc-start 100 20220330094646.191 NOTICE   utils - utils.c:lxc_setgroups:1472 - Dropped additional groups
lxc-start 100 20220330094646.191 WARN     cgfsng - cgroups/cgfsng.c:fchowmodat:1548 - No such file or directory - Failed to fchownat(29, memory.oom.group, 65536, 0, AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW )

Another thing I notice : the owner of /proc and /sys is not root. I suppose this is the cause of all my problems. How can I correct this please ?

Code: 
# ls -ld /proc
dr-xr-xr-x 435 65534 65534 0 30 mars  12:04 /proc
# ls -ld /sys
dr-xr-xr-x 13 65534 65534 0 30 mars  12:04 /sys
 
no, that's normal in an unprivileged container. please clearly describe the actual issues you are seeing - else you won't be able to get any help..
 
The CT is unreachable (no ssh, no ping, not a firewall problem).

When I restart network I get :

Code: 
# service network restart
/etc/init.d/functions: ligne63: /dev/stderr: Permission non accordée
/etc/init.d/functions: line 63: /dev/stderr: Permission denied
Restarting network (via systemctl):                        [  OK  ]

It returns OK but still no network ("network is unreachable" if I try to ping from inside CT).
 
Last edited:
then check the network configuration inside the CT - my guess is you have some leftover/broken stuff from the conversion from OpenVZ in there..

e.g., what does ip a and ip l inside the CT say? what do the network config files contain?
 
fabian said:
then check the network configuration inside the CT - my guess is you have some leftover/broken stuff from the conversion from OpenVZ in there..

e.g., what does ip a and ip l inside the CT say? what do the network config files contain?
Click to expand...
Code: 
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if105: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 72:79:b0:72:57:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 62.xxx.yyy.zzz/24 brd 62.xxx.yyy.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::7079:b0ff:fe72:57ff/64 scope link
       valid_lft forever preferred_lft forever


Code: 
# ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0@if105: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 72:79:b0:72:57:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0

Whatever the ip address I mount it doesn't work as described at #27.
 
well, the interface is up and has an IP configured (the one you set in the container config? not clear since you censored in a different fashion both times..). if you can't reach the internet from within the container, regular network trouble shooting applies (can you ping the gateway? the hypervisor? firewall active anywhere? ...)
 
what does ip r say? and again, I'd recommend checking the network configuration inside the container for any leftover cruft that is not managed by PVE..
 
ip r returns same thing that other CT that works correctly. Something is broken in the CT and I can't repair it. All errors I get when I run basic commands like below make me hopeless. I thought maybe rights on a directory had to be corrected somewhere... I have to resign to lose the 2 CT :(
Code: 
# ifdown eth0
/etc/init.d/functions: ligne63: /dev/stderr: Permission non accordée
/etc/init.d/functions: ligne63: /dev/stderr: Permission non accordée
/etc/init.d/functions: ligne63: /dev/stderr: Permission non accordée
/etc/init.d/functions: ligne63: /dev/stderr: Permission non accordée
 
what's at line 63 of /etc/init.d/functions ? can you do stat /dev/stderr inside the container? could you add the same cmd right before line 63 in /etc/init.d/functions and repeat one of the invocations that print that error?
 
The line 63 is in bold below :

if [ -z "${CONSOLETYPE:-}" ]; then
if [ -c "/dev/stderr" -a -r "/dev/stderr" ]; then
CONSOLETYPE="$(/sbin/consoletype < /dev/stderr 2>/dev/null)"
else
CONSOLETYPE="serial"
fi

Code: 
# stat /dev/stderr
  Fichier : « /dev/stderr » -> « /proc/self/fd/2 »
   Taille : 15          Blocs : 0          Blocs d'E/S : 4096   lien symbolique
Périphérique : 4ch/76d  Inode : 17          Liens : 1
Accès : (0777/lrwxrwxrwx)  UID : (    0/    root)   GID : (    0/    root)
 Accès : 2022-03-31 11:36:27.236283997 +0200
Modif. : 2022-03-31 11:36:27.036285931 +0200
Changt : 2022-03-31 11:36:27.036285931 +0200
  Créé : -

/proc/self/fd/2 is link to /dev/pts/2

Code: 
# ls -l /dev/pts/*
crw--w---- 1 root  tty  136, 0 31 mars  11:36 /dev/pts/0
crw--w---- 1 root  tty  136, 1 31 mars  11:36 /dev/pts/1
crw--w---- 1 65534 tty  136, 2 31 mars  11:39 /dev/pts/2
crw-rw-rw- 1 root  root   5, 2 31 mars  11:39 /dev/pts/ptmx

The owner of /dev/pts/2 is 65534...
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom