LXC Clone - Total file size incorrect

L

lcasale

Renowned Member
Oct 19, 2012
10
0
66
I'm trying to have a base LXC container preconfigured in my homelab (smb, SSD, rsyslog, etc all preconfigured) to use as a template any time I need to spin up a new container. However, when I clone I try to full-clone the container, it's taking much longer than it should. And instead of writing only ~8GB that I would expect, it's writing ~465GB.

Container Config
Code: 
arch: amd64
cores: 1
hostname: unifivideo
memory: 1024
net0: name=eth0,bridge=vmbr1,firewall=1,hwaddr=0E:CC:FC:21:B4:68,ip=dhcp,tag=33,type=veth
ostype: ubuntu
rootfs: local-thin:vm-102-disk-0,size=8G
swap: 0
unprivileged: 1
lxc.idmap: u 0 100000 2000000000
lxc.idmap: g 0 100000 2000000000

Logs
Code: 
Virtual Environment 5.4-13
Search
Node 'proxmox'
Server View
Logs
()   
create full clone of mountpoint rootfs (local-thin:vm-102-disk-0)
  Using default stripesize 64.00 KiB.
  Logical volume "vm-110-disk-0" created.
mke2fs 1.43.4 (31-Jan-2017)
Discarding device blocks:    4096/2097152               done                           
Creating filesystem with 2097152 4k blocks and 524288 inodes
Filesystem UUID: 78faee09-e98a-48ad-b5e0-0bf8b3210389
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables:  0/64     done                           
Writing inode tables:  0/64     done                           
Creating journal (16384 blocks): done
Multiple mount protection is enabled with update interval 5 seconds.
Writing superblocks and filesystem accounting information:  0/64     done


Number of files: 25,856 (reg: 19,865, dir: 2,408, link: 3,536, special: 47)
Number of created files: 25,854 (reg: 19,865, dir: 2,406, link: 3,536, special: 47)
Number of deleted files: 0
Number of regular files transferred: 19,846
Total file size: 465,304,020,289 bytes
Total transferred file size: 465,091,247,727 bytes
Literal data: 465,091,247,727 bytes
Matched data: 0 bytes
File list size: 917,436
File list generation time: 0.004 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 465,206,367,485
Total bytes received: 402,286

sent 465,206,367,485 bytes  received 402,286 bytes  325,889,155.71 bytes/sec
total size is 465,304,020,289  speedup is 1.00
TASK OK

Disk config
1586369705559.png

Any idea why this is happening?
 
can you post the output of 'lvs' ?
 
Sure, it's...
Code: 
  LV              VG         Attr       LSize    Pool       Origin Data%  Meta%  Move Log Cpy%Sync Convert
  base-200-disk-0 local-thin Vri-a-tz-k    8.00g local-thin        19.99                                 
  local-thin      local-thin twi-aotz-- <222.78g                   26.28  22.60                           
  vm-100-disk-0   local-thin Vwi-a-tz--   32.00g local-thin        8.60                                   
  vm-102-disk-0   local-thin Vwi-a-tz--    8.00g local-thin        20.58                                 
  vm-104-disk-0   local-thin Vwi-aotz--   16.00g local-thin        93.46                                 
  vm-108-disk-0   local-thin Vwi-a-tz--    8.00g local-thin        14.60                                 
  vm-110-disk-0   local-thin Vwi-a-tz--    8.00g local-thin        14.95                                 
  vm-113-disk-0   local-thin Vwi-a-tz--    8.00g local-thin        14.95                                 
  vm-115-disk-0   local-thin Vwi-aotz--    8.00g local-thin        92.35                                 
  vm-121-disk-0   local-thin Vwi-aotz--   32.00g local-thin        31.18                                 
  vm-122-disk-1   local-thin Vwi-aotz--   16.00g local-thin        90.48                                 
  vm-123-disk-0   local-thin Vwi-a-tz--    8.00g local-thin        12.67                                 
  vm-201-disk-0   local-thin Vwi-a-tz--    8.00g local-thin        14.83                                 
  vm-106-disk-0   plex       -wi-ao----  148.00g                                                         
  root            pve        -wi-ao----  342.65g                                                         
  swap            pve        -wi-ao----    8.00g
 
Any ideas? Nothing from the 'lvs' output gives any indication of a problem to me.

I'm toying with the idea of blowing up the installation and creating a ZFS pool for all the storage if I can't get to the bottom of this.
 
mhm... not really an idea... altough the target disk is the correct size, i would not know where it should write that much data
what does a 'mount' and 'df -h' inside the container show?
can you post the logs (syslog/dmesg) from the time of the clone?
 
I ended up nuking my old Proxmox install and started fresh with local-zfs storage. Creating a brand new Ubuntu container with 8GB storage and adding/configuring some packages STILL is writing 465GB somewhere when cloning.

Could lxc.idmap be causing it? Or something configured within Active Directory after the container joins the domain?

Syslog
Code: 
May  2 16:51:14 franky pct[25999]: <root@pam> starting task UPID:franky:000065A1:005B8259:5EADDD42:vzclone:200:root@pam:
May  2 16:51:14 franky zed: eid=40 class=history_event pool_guid=0x59A1A8BF35BF10E4 
May  2 16:51:15 franky zed: eid=41 class=history_event pool_guid=0x59A1A8BF35BF10E4 
May  2 16:51:15 franky zed: eid=42 class=history_event pool_guid=0x59A1A8BF35BF10E4 
May  2 16:51:15 franky zed: eid=43 class=history_event pool_guid=0x59A1A8BF35BF10E4 
May  2 16:52:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 16:52:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 16:52:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 16:53:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 16:53:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 16:53:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 16:54:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 16:54:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 16:54:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 16:55:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 16:55:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 16:55:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 16:56:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 16:56:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 16:56:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 16:56:20 franky pveproxy[5633]: worker exit
May  2 16:56:20 franky pveproxy[2932]: worker 5633 finished
May  2 16:56:20 franky pveproxy[2932]: starting 1 worker(s)
May  2 16:56:20 franky pveproxy[2932]: worker 3348 started
May  2 16:57:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 16:57:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 16:57:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 16:58:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 16:58:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 16:58:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 16:59:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 16:59:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 16:59:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 16:59:22 franky pveproxy[10710]: worker exit
May  2 16:59:22 franky pveproxy[2932]: worker 10710 finished
May  2 16:59:22 franky pveproxy[2932]: starting 1 worker(s)
May  2 16:59:22 franky pveproxy[2932]: worker 15952 started
May  2 17:00:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:00:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:00:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:01:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:01:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:01:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:01:40 franky pvedaemon[14777]: <root@pam> successful auth for user 'root@pam'
May  2 17:02:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:02:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:02:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:03:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:03:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:03:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:04:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:04:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:04:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:05:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:05:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:05:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:06:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:06:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:06:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:07:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:07:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:07:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:08:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:08:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:08:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:09:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:09:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:09:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:10:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:10:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:10:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:11:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:11:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:11:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:12:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:12:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:12:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:13:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:13:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:13:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:14:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:14:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:14:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:15:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:15:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:15:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:16:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:16:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:16:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:16:41 franky pvedaemon[14777]: <root@pam> successful auth for user 'root@pam'
May  2 17:17:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:17:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:17:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:17:01 franky CRON[44164]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
May  2 17:18:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:18:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:18:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:19:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:19:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:19:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:20:00 franky systemd[1]: Starting Proxmox VE replication runner...
May  2 17:20:01 franky systemd[1]: pvesr.service: Succeeded.
May  2 17:20:01 franky systemd[1]: Started Proxmox VE replication runner.
May  2 17:20:15 franky pct[25999]: <root@pam> end task UPID:franky:000065A1:005B8259:5EADDD42:vzclone:200:root@pam: OK
 
mount
Code: 
root@template-lxc:~# mount
rpool/data/subvol-200-disk-0 on / type zfs (rw,noatime,xattr,posixacl)
none on /dev type tmpfs (rw,relatime,size=492k,mode=755,uid=100000,gid=100000)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
sysfs on /sys/devices/virtual/net type sysfs (rw,relatime)
sysfs on /sys/devices/virtual/net type sysfs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/loadavg type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/swaps type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /sys/devices/system/cpu/online type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
udev on /dev/full type devtmpfs (rw,nosuid,relatime,size=98991524k,nr_inodes=24747881,mode=755)
udev on /dev/null type devtmpfs (rw,nosuid,relatime,size=98991524k,nr_inodes=24747881,mode=755)
udev on /dev/random type devtmpfs (rw,nosuid,relatime,size=98991524k,nr_inodes=24747881,mode=755)
udev on /dev/tty type devtmpfs (rw,nosuid,relatime,size=98991524k,nr_inodes=24747881,mode=755)
udev on /dev/urandom type devtmpfs (rw,nosuid,relatime,size=98991524k,nr_inodes=24747881,mode=755)
udev on /dev/zero type devtmpfs (rw,nosuid,relatime,size=98991524k,nr_inodes=24747881,mode=755)
devpts on /dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=1024)
devpts on /dev/ptmx type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=1024)
devpts on /dev/tty1 type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=1024)
devpts on /dev/tty2 type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=1024)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,uid=100000,gid=100000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,size=19803172k,mode=755,uid=100000,gid=100000)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,uid=100000,gid=100000)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755,uid=100000,gid=100000)
cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
tmpfs on /run/user/1589201105 type tmpfs (rw,nosuid,nodev,relatime,size=19803168k,mode=700,uid=1589301105,gid=1589300513)

df -h
Code: 
root@template-lxc:~# df -h
Filesystem                    Size  Used Avail Use% Mounted on
rpool/data/subvol-200-disk-0  8.0G  479M  7.6G   6% /
none                          492K     0  492K   0% /dev
udev                           95G     0   95G   0% /dev/tty
tmpfs                          95G     0   95G   0% /dev/shm
tmpfs                          19G  140K   19G   1% /run
tmpfs                         5.0M     0  5.0M   0% /run/lock
tmpfs                          95G     0   95G   0% /sys/fs/cgroup
tmpfs                          19G     0   19G   0% /run/user/1589201105

200.conf
Code: 
root@franky:/etc/pve/lxc# cat 200.conf
arch: amd64
cores: 2
hostname: template-lxc
lock: disk
memory: 1024
net0: name=eth0,bridge=vmbr1,firewall=1,hwaddr=AA:FE:EB:E1:39:BA,ip=dhcp,tag=33,type=veth
ostype: ubuntu
parent: JoinedAD
rootfs: local-zfs:subvol-200-disk-0,size=8G
swap: 512
unprivileged: 1
lxc.idmap: u 0 100000 2000000000
lxc.idmap: g 0 100000 2000000000
 
can you please also send the task log that corresponds to the dmesg output above?
 
Sorry yes, here it is.

Code: 
create full clone of mountpoint rootfs (local-zfs:subvol-200-disk-0)

Number of files: 25,179 (reg: 19,459, dir: 2,454, link: 3,228, special: 38)
Number of created files: 25,178 (reg: 19,459, dir: 2,453, link: 3,228, special: 38)
Number of deleted files: 0
Number of regular files transferred: 19,440
Total file size: 465,194,237,171 bytes
Total transferred file size: 464,951,621,141 bytes
Literal data: 464,951,621,141 bytes
Matched data: 0 bytes
File list size: 917,409
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 465,066,667,261
Total bytes received: 393,782

sent 465,066,667,261 bytes  received 393,782 bytes  267,049,704.88 bytes/sec
total size is 465,194,237,171  speedup is 1.00
TASK OK
 
mhmm... ok. honestly i am stumped.. if you did not mount anything in the container (and your mount/df output indicates that you did not mount anything special) i have really no idea where this could come from (or better where it writes it to...)
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom