Problems with nfs (Openindiana ZFS Server) and container creation / migration

A

acw-flo

New Member
Feb 3, 2012
9
0
1
Hi,

I set up three Server with Proxmox 2 rc and made a cluster with them.
The Servers are IBM HS21 Blades
I updated them to the final version with aptitude update and aptitude full-upgrade:

For all three nodes:
Code: 
pveversion --verbose
pve-manager: 2.0-54 (pve-manager/2.0/4b59ea39)
running kernel: 2.6.32-10-pve
proxmox-ve-2.6.32: 2.0-63
pve-kernel-2.6.32-10-pve: 2.6.32-63
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve2
clvm: 2.02.88-2pve2
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-33
pve-firmware: 1.0-15
libpve-common-perl: 1.0-23
libpve-access-control: 1.0-17
libpve-storage-perl: 2.0-16
vncterm: 1.0-2
vzctl: 3.0.30-2pve2
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-8
ksm-control-daemon: 1.1-1


Code: 
mount
/dev/mapper/pve-data on /var/lib/vz type ext3 (rw)
beancounter on /proc/vz/beancounter type cgroup (rw,name=beancounter)
container on /proc/vz/container type cgroup (rw,name=container)
fairsched on /proc/vz/fairsched type cgroup (rw,name=fairsched)
...
10.199.199.1:/tank/proxmox_nfs on /mnt/pve/proxmox_nfs type nfs (rw,vers=3,addr=10.199.199.1)


Code: 
cat /etc/pve/storage.cfg 
dir: local
    path /var/lib/vz
    content images,iso,vztmpl,rootdir

nfs: proxmox_nfs
    path /mnt/pve/proxmox_nfs
    server 10.199.199.1
    export /tank/proxmox_nfs
    options vers=3
    content iso,vztmpl,backup,rootdir
    maxfiles 1


Code: 
pvecm nodes
Node  Sts   Inc   Joined               Name
   1   M    104   2012-04-04 11:13:23  proxmox1
   2   M    120   2012-04-04 11:13:23  proxmox2
   3   M    120   2012-04-04 11:13:23  proxmox3


Here is the zfs nfs share config:
Code: 
zfs get sharenfs tank/proxmox_nfs
NAME              PROPERTY  VALUE                                                SOURCE
tank/proxmox_nfs  sharenfs  rw,root=@10.199.199.0/24:proxmox1:proxmox2:proxmox3  local


The first problem:
Creation of a container on nfs.
Code: 
Creating container private area (/mnt/pve/proxmox_nfs/template/cache/debian-6.0-amd64-minimal.tar.gz)
Performing postcreate actions
/bin/cp: preserving permissions for `/var/lib/vz/root/103/etc/crontab.507520': Operation not supported
CT configuration saved to /etc/pve/openvz/103.conf
Container private area was created
TASK OK


The second problem:
The init.log and init.fifo files have permission problems:
Code: 
ls -alh /mnt/pve/proxmox_nfs/private/100/var/log/init.log
---s--S--T+ 1 root root 0 Apr  4 11:26 /mnt/pve/proxmox_nfs/private/100/var/log/init.log


cat /mnt/pve/proxmox_nfs/private/100/var/log/init.log
cat: /mnt/pve/proxmox_nfs/private/100/var/log/init.log: Permission denied


The migration offline and online fail due to the "Permission denied" problem:
Code: 
Apr 03 13:47:28 starting migration of CT 103 to node 'proxmox2' (172.16.2.188)
Apr 03 13:47:28 container is running - using online migration
Apr 03 13:47:28 container data is on shared storage 'proxmox_nfs'
Apr 03 13:47:28 start live migration - suspending container
Apr 03 13:47:28 dump container state
Apr 03 13:47:28 dump 2nd level quota
Apr 03 13:47:30 initialize container on remote node 'proxmox2'
Apr 03 13:47:30 initializing remote quota
Apr 03 13:47:30 # /usr/bin/ssh -c blowfish -o 'BatchMode=yes' root@172.16.2.188 vzctl quotainit 103
Apr 03 13:47:30 vzquota : (error) Quota check : open 'init.log': Permission denied
Apr 03 13:47:30 ERROR: online migrate failure - Failed to initialize quota: vzquota init failed [1]
Apr 03 13:47:30 start final cleanup
Apr 03 13:47:30 ERROR: migration finished with problems (duration 00:00:02)
TASK ERROR: migration problems


After the mirgation, I have to delete both init files and then the container will start, but in the webinterface, the InitLog tab shows:
Code: 
unable to open file - Permission denied


With a container on the local storage I don't have such problems.

What can I do to solve these problems?

Regards Flo
 
Hi,

I did some further investigations and there seems to be, that if I change the permission of the file init.log to 666, then I can offline migrate without problems and the init.log file is used to log what it should do.

Why has this log-file these permissions?
Can I change the permissions in the configuration?

Regards Flo
 
Hi,

i probably found the solution. I had to disable the suid feature in the nfs Server config.
With
Code: 
zfs set sharenfs=rw,noaclfab,root_mapping=0,nosuid,root=@10.199.199.0/24:proxmox1:proxmox2:proxmox3 tank/proxmox_nfs
I solved the problem.
Now the file permission is like this:
Code: 
[LEFT][COLOR=#3E3E3E]---x-----T 1 root root 0 Apr  4 11:26 /mnt/pve/proxmox_nfs/private/100/var/log/init.log[/COLOR][/LEFT]
No SUID Bit anymore, but the file is writable and offline migration is working.

Online migration fails with the error reported in the bug 71

Regards Flo
 
You must log in or register to reply here.
Top Bottom