mysql installation in lxc container failed

ness1602 · Apr 7, 2022

Yes,my output is same as ik777, i will wait his ct config

ness1602 · Apr 7, 2022

Bash:

arch: amd64
cores: 2
features: nesting=1
hostname: zabbix2ubuntu
memory: 2048
net0: name=eth0,bridge=vmbr0,firewall=1,gw=private,hwaddr=private,ip=private,tag=111,type=veth
ostype: ubuntu
rootfs: zdata:subvol-127-disk-0,size=16G
swap: 512
unprivileged: 1

oguz · Apr 7, 2022

ness1602 said:

Bash:

arch: amd64
cores: 2
features: nesting=1
hostname: zabbix2ubuntu
memory: 2048
net0: name=eth0,bridge=vmbr0,firewall=1,gw=private,hwaddr=private,ip=private,tag=111,type=veth
ostype: ubuntu
rootfs: zdata:subvol-127-disk-0,size=16G
swap: 512
unprivileged: 1

thanks for the config. could you check /proc/cmdline on your PVE host as well?

ness1602 · Apr 7, 2022

Code:

 cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.15.30-1-pve root=/dev/mapper/pve-root ro quiet

lk777 · Apr 7, 2022

pve host:

Code:

BOOT_IMAGE=/boot/vmlinuz-5.15.30-1-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on

lxc:

Code:

BOOT_IMAGE=/boot/vmlinuz-5.15.30-1-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on

Code:

arch: amd64
cores: 2
features: nesting=1
hostname: ubuntu-lxc-3
memory: 2048
net0: name=eth0,bridge=vmbr0,firewall=1,gw=10.10.5.1,hwaddr=E6:F9:07:42:05:4F,ip=10.10.5.93/24,type=veth
ostype: ubuntu
rootfs: zfsa:subvol-202-disk-0,size=10G
swap: 2028
unprivileged: 1

lk777 · Apr 7, 2022

@oguz,

What CPU model do you have on your pve host? Mine is AMD Ryzen.

oguz · Apr 7, 2022

lk777 said:
@oguz,

What CPU model do you have on your pve host? Mine is AMD Ryzen.

also running a ryzen here so i think that's not the issue...

ness1602 · Apr 7, 2022

Yeah ,on my side it is intel,older one.

oguz · Apr 7, 2022

does mysql installation work on a newer ubuntu template?

could you try with 21.04 and 21.10 as well and let us know here?

for the record still cannot reproduce the issue...

ness1602 · Apr 7, 2022

but those templates arent LTS, they are of no use to us.

lk777 · Apr 7, 2022

For the test purposes I have created a container based on these images (rootfs.tar.xz) with the same results.
Launched a lxd container ubuntu:20.04 on ubuntu 20.04 host and no issues with mysql.

I am just guessing, can it be zfs/subvol that drives MySQL InnoDB (Ubuntu) crazy? My Ubuntu host is on btrfs.
It is definitely MariaDB (Ubuntu) better handles InnoDB on ZFS, no issues so far.

In all my working containers (on ubuntu host with btrfs) and MariaDB on zfs (proxmox) I have:

Code:

-rw-rw----  1 mysql mysql 50331648 Apr  7 17:43 ib_logfile0
-rw-rw----  1 mysql mysql 50331648 Apr  7 17:43 ib_logfile1

And this on the proxmox CT (MySQL):

Code:

-rw-rw----  1 mysql mysql 17825792  Apr  7 17:43 ib_logfile0
-rw-rw----  1 mysql mysql 0         Apr  7 17:43 ib_logfile1

This difference in size causes those issues.

What files system do you have on your PVE?

lk777 · Apr 8, 2022

UPDATE:

@oguz

Code:

rootfs: local:137/vm-137-disk-0.raw,size=8G

@ness1602

Code:

rootfs: zdata:subvol-127-disk-0,size=16G

mine:

Code:

rootfs: zfsa:subvol-202-disk-0,size=10G

So this is where we have differences.

h0tw1r3 · Apr 13, 2022

Same issue here. Using 20.04 template.

arch: amd64
cmode: tty
console: 1
cores: 2
cpulimit: 0
cpuunits: 1024
features: nesting=1
hostname: onlyoffice
memory: 6144
net0: name=eth0,bridge=vmbr0,gw=192.168.47.254,hwaddr=7E:AA:B7:B2:E3:6A,ip=192.168.47.25/24,type=veth
onboot: 0
ostype: ubuntu
protection: 0
rootfs: local-zfs:subvol-106-disk-0,size=20G
swap: 0
tty: 2
unprivileged: 1

Matthias. · Apr 13, 2022

I can reproduce the problem with a privileged container on a ZFS storage. So that seems to be the issue

h0tw1r3 · Apr 14, 2022

Exact same problem. Only occurs when using local zfs storage, privileged or unprivileged. Searching for a fix.

h0tw1r3 · Apr 15, 2022

impacts percona-server 8.0 (mysql fork) also.

If I add the following to the server config:

Code:

[mysqld]
innodb_log_file_size = 4194304
innodb_page_size = 65536

I can get the server to start.
Magic is in the increasing of the innodb_page_size. Smells like a zfs bug.

ETNY · May 7, 2022

h0tw1r3 said:
impacts percona-server 8.0 (mysql fork) also.

If I add the following to the server config:

Code:

[mysqld] innodb_log_file_size = 4194304 innodb_page_size = 65536

I can get the server to start.
Magic is in the increasing of the innodb_page_size. Smells like a zfs bug.

Hello,
Excuse me for a basic question but where you put those lines?

Usama Ahmad · May 10, 2022

https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1969247

kamzata · May 15, 2022

h0tw1r3 said:
impacts percona-server 8.0 (mysql fork) also.

If I add the following to the server config:

Code:

[mysqld] innodb_log_file_size = 4194304 innodb_page_size = 65536

I can get the server to start.
Magic is in the increasing of the innodb_page_size. Smells like a zfs bug.

It didn't work for me on Proxmox 7.2-3 with ZFS and LXC with Ubuntu 22.04 with mysql-server.

Bash:

...
Setting up mysql-server-8.0 (8.0.29-0ubuntu0.22.04.2) ...
Renaming removed key_buffer and myisam-recover options (if present)
Warning: Unable to start the server.
Job for mysql.service failed because the control process exited with error code.
See "systemctl status mysql.service" and "journalctl -xeu mysql.service" for details.
invoke-rc.d: initscript mysql, action "start" failed.
● mysql.service - MySQL Community Server
     Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Sat 2022-05-14 13:58:15 CEST; 3ms ago
    Process: 5647 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
    Process: 5655 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
   Main PID: 5655 (code=exited, status=1/FAILURE)
     Status: "Server startup in progress"
      Error: 22 (Invalid argument)
        CPU: 280ms

May 14 13:58:15 stats systemd[1]: Failed to start MySQL Community Server.
dpkg: error processing package mysql-server-8.0 (--configure):
 installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of mysql-server:
 mysql-server depends on mysql-server-8.0; however:
  Package mysql-server-8.0 is not configured yet.

dpkg: error processing package mysql-server (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 mysql-server-8.0
 mysql-server
E: Sub-process /usr/bin/dpkg returned an error code (1)

This is the status output:

Bash:

 ⌁ root  ~  service mysql status
× mysql.service - MySQL Community Server
     Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sat 2022-05-14 13:58:19 CEST; 3min 31s ago
    Process: 5794 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
    Process: 5802 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
   Main PID: 5802 (code=exited, status=1/FAILURE)
     Status: "Server startup in progress"
      Error: 22 (Invalid argument)
        CPU: 277ms

May 14 13:58:19 stats systemd[1]: Failed to start MySQL Community Server.
May 14 13:58:19 stats systemd[1]: mysql.service: Scheduled restart job, restart counter is at 5.
May 14 13:58:19 stats systemd[1]: Stopped MySQL Community Server.
May 14 13:58:19 stats systemd[1]: mysql.service: Start request repeated too quickly.
May 14 13:58:19 stats systemd[1]: mysql.service: Failed with result 'exit-code'.
May 14 13:58:19 stats systemd[1]: Failed to start MySQL Community Server.

and this is the /var/log/mysql/error.log file:

Code:

2022-05-14T01:14:09.861441Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.29-0ubuntu0.22.04.2) initializing of server in progress as process 794
2022-05-14T01:14:09.867136Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-05-14T01:14:10.094521Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2022-05-14T01:14:11.220254Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2022-05-14T01:14:12.079700Z 6 [System] [MY-013172] [Server] Received SHUTDOWN from user boot. Shutting down mysqld (Version: 8.0.29-0ubuntu0.22.04.2).
2022-05-14T01:14:15.266629Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.29-0ubuntu0.22.04.2) starting as process 844
2022-05-14T01:14:15.278804Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-05-14T01:14:15.298511Z 1 [ERROR] [MY-012963] [InnoDB] Log file ./ib_logfile1 is of different size 0 bytes than other log files 18874368 bytes!
2022-05-14T01:14:15.298551Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error.
2022-05-14T01:14:15.789792Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
2022-05-14T01:14:15.789963Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2022-05-14T01:14:15.790050Z 0 [ERROR] [MY-010119] [Server] Aborting
2022-05-14T01:14:15.790748Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.29-0ubuntu0.22.04.2)  (Ubuntu).

kamzata · May 15, 2022

h0tw1r3 said:
impacts percona-server 8.0 (mysql fork) also.

If I add the following to the server config:

Code:

[mysqld] innodb_log_file_size = 4194304 innodb_page_size = 65536

I can get the server to start.
Magic is in the increasing of the innodb_page_size. Smells like a zfs bug.

I tried using MariaDB as well but it still doesn't work. Same issue.

mysql installation in lxc container failed

Renowned Member

Renowned Member

Proxmox Retired Staff

Renowned Member

Member

Member

Proxmox Retired Staff

Renowned Member

Proxmox Retired Staff

Renowned Member

Member

Member

Renowned Member

Proxmox Retired Staff

Renowned Member

Renowned Member

Member

Active Member

Renowned Member

Renowned Member