Problems with nfs (Openindiana ZFS Server) and container creation / migration

[acw-flo]

Hi,

I set up three Server with Proxmox 2 rc and made a cluster with them.
The Servers are IBM HS21 Blades
I updated them to the final version with aptitude update and aptitude full-upgrade:

For all three nodes:

pveversion --verbose
pve-manager: 2.0-54 (pve-manager/2.0/4b59ea39)
running kernel: 2.6.32-10-pve
proxmox-ve-2.6.32: 2.0-63
pve-kernel-2.6.32-10-pve: 2.6.32-63
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve2
clvm: 2.02.88-2pve2
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-33
pve-firmware: 1.0-15
libpve-common-perl: 1.0-23
libpve-access-control: 1.0-17
libpve-storage-perl: 2.0-16
vncterm: 1.0-2
vzctl: 3.0.30-2pve2
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-8
ksm-control-daemon: 1.1-1

mount
/dev/mapper/pve-data on /var/lib/vz type ext3 (rw)
beancounter on /proc/vz/beancounter type cgroup (rw,name=beancounter)
container on /proc/vz/container type cgroup (rw,name=container)
fairsched on /proc/vz/fairsched type cgroup (rw,name=fairsched)
...
10.199.199.1:/tank/proxmox_nfs on /mnt/pve/proxmox_nfs type nfs (rw,vers=3,addr=10.199.199.1)

cat /etc/pve/storage.cfg 
dir: local
    path /var/lib/vz
    content images,iso,vztmpl,rootdir

nfs: proxmox_nfs
    path /mnt/pve/proxmox_nfs
    server 10.199.199.1
    export /tank/proxmox_nfs
    options vers=3
    content iso,vztmpl,backup,rootdir
    maxfiles 1

pvecm nodes
Node  Sts   Inc   Joined               Name
   1   M    104   2012-04-04 11:13:23  proxmox1
   2   M    120   2012-04-04 11:13:23  proxmox2
   3   M    120   2012-04-04 11:13:23  proxmox3

Here is the zfs nfs share config:

zfs get sharenfs tank/proxmox_nfs
NAME              PROPERTY  VALUE                                                SOURCE
tank/proxmox_nfs  sharenfs  rw,root=@10.199.199.0/24:proxmox1:proxmox2:proxmox3  local

The first problem:
Creation of a container on nfs.

Creating container private area (/mnt/pve/proxmox_nfs/template/cache/debian-6.0-amd64-minimal.tar.gz)
Performing postcreate actions
/bin/cp: preserving permissions for `/var/lib/vz/root/103/etc/crontab.507520': Operation not supported
CT configuration saved to /etc/pve/openvz/103.conf
Container private area was created
TASK OK

The second problem:
The init.log and init.fifo files have permission problems:

ls -alh /mnt/pve/proxmox_nfs/private/100/var/log/init.log
---s--S--T+ 1 root root 0 Apr  4 11:26 /mnt/pve/proxmox_nfs/private/100/var/log/init.log


cat /mnt/pve/proxmox_nfs/private/100/var/log/init.log
cat: /mnt/pve/proxmox_nfs/private/100/var/log/init.log: Permission denied

The migration offline and online fail due to the "Permission denied" problem:

Apr 03 13:47:28 starting migration of CT 103 to node 'proxmox2' (172.16.2.188)
Apr 03 13:47:28 container is running - using online migration
Apr 03 13:47:28 container data is on shared storage 'proxmox_nfs'
Apr 03 13:47:28 start live migration - suspending container
Apr 03 13:47:28 dump container state
Apr 03 13:47:28 dump 2nd level quota
Apr 03 13:47:30 initialize container on remote node 'proxmox2'
Apr 03 13:47:30 initializing remote quota
Apr 03 13:47:30 # /usr/bin/ssh -c blowfish -o 'BatchMode=yes' root@172.16.2.188 vzctl quotainit 103
Apr 03 13:47:30 vzquota : (error) Quota check : open 'init.log': Permission denied
Apr 03 13:47:30 ERROR: online migrate failure - Failed to initialize quota: vzquota init failed [1]
Apr 03 13:47:30 start final cleanup
Apr 03 13:47:30 ERROR: migration finished with problems (duration 00:00:02)
TASK ERROR: migration problems

After the mirgation, I have to delete both init files and then the container will start, but in the webinterface, the InitLog tab shows:

unable to open file - Permission denied

With a container on the local storage I don't have such problems.

What can I do to solve these problems?

Regards Flo

 

[tom]

seems you have a permission issue on your nfs server, dig deeper there.

online migration: see https://bugzilla.proxmox.com/show_bug.cgi?id=71

 

[acw-flo]

Hi,

I did some further investigations and there seems to be, that if I change the permission of the file init.log to 666, then I can offline migrate without problems and the init.log file is used to log what it should do.

Why has this log-file these permissions?
Can I change the permissions in the configuration?

Regards Flo

 

[acw-flo]

Hi,

i probably found the solution. I had to disable the suid feature in the nfs Server config.
With

zfs set sharenfs=rw,noaclfab,root_mapping=0,nosuid,root=@10.199.199.0/24:proxmox1:proxmox2:proxmox3 tank/proxmox_nfs

I solved the problem.
Now the file permission is like this:

[LEFT][COLOR=#3E3E3E]---x-----T 1 root root 0 Apr  4 11:26 /mnt/pve/proxmox_nfs/private/100/var/log/init.log[/COLOR][/LEFT]

No SUID Bit anymore, but the file is writable and offline migration is working.

Online migration fails with the error reported in the bug 71

Regards Flo