CT in glusterfs

mir

mir

Famous Member
Apr 14, 2012
3,585
142
133
Copenhagen, Denmark
Hi all,

There seems to be either a bug or a limitation when installing a CT in a gluster filesystem.

Background:
Creating a folder in /var/lib/vz on to nodes forming a replica 2 gluster volume (/var/lib/vz is a ext4 filesystem over LVM)
In this volume creating a i386 CT utterly fails given numerous errors like this one: run-parts: failed to open directory /etc/network/if-down.d: Value too large for defined data type.

using exact same template but a amd64 instead works as expected.

My question: Is this a bug or a feature?
 
There is no problem installing it. The errors first show up when you start the CT. The error message is from the CT when booting. I have tried two different turnkey templates and the standard debian 7 template available from the list of templates in proxmox.
 
mir said:
There is no problem installing it. The errors first show up when you start the CT. The error message is from the CT when booting. I have tried two different turnkey templates and the standard debian 7 template available from the list of templates in proxmox.
Click to expand...
I'm interested in this solution too. A note: the Gluster devs recommend xfs with a large enough inode size (-i size=512), I'd recommend trying with xfs. What Gluster version are you using? 4.2 or 5.x? Trying with the other instead of your current one might also worth a try.
 
I will try your suggestion with xfs but your other suggestion of upgrading glusterfs to 5.x is IMHO not an option. An enterprise setup should always use the packages from the distribution.
 
mir said:
I will try your suggestion with xfs but your other suggestion of upgrading glusterfs to 5.x is IMHO not an option. An enterprise setup should always use the packages from the distribution.
Click to expand...
I mainly agree, though it's up to debate what constitutes "enterprise setup" and "distribution". I think it never hurts to try and test.
 
mir said:
I will try your suggestion with xfs but your other suggestion of upgrading glusterfs to 5.x is IMHO not an option. An enterprise setup should always use the packages from the distribution.
Click to expand...
Just tried using xfs but same error messages.

Code: 
xfs_info /dev/mapper/pve-gfs01 
meta-data=/dev/mapper/pve-gfs01  isize=512    agcount=4, agsize=1966080 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=7864320, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=3840, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
 
After doing a quick test and getting the same error as you, I started to dig the net a little and again some bad news came up. All sources point to the fact that many 32-bit binaries are not built with support for 64-bit inode numbers. A workaround option exists for NFS clients, see here for example. So it looks that by using the builtin NFS server of Gluster, and setting an option exists as a workaround for x86 containers. Or one should always create 64-bit ones. Nowadays the larger memory footprint of 64-bit apps are not that much of an issue on a modern server. It doesn't help when importing existing x86 templates, though.

The NFS option is:
Code: 
gluster vol set volname nfs.enable-ino32 on
 
You must log in or register to reply here.
Top Bottom