sshfs shared storage

**bread-baker** · 05-30-2012

I think it would be good to allow sshfs for shared storage.

For us sshfs is more reliable then nfs . We use sshfs for user documents - with $HOME/Documents targeted to a kvm sshfs . We've done that for 3 years and have had very few issues.

**tom** · 05-30-2012

never tried but just mount via /etc/fstab and then configure a shared directory storage, does that work for you?

**bread-baker** · 05-30-2012

the mount and share work, thanks for the idea.

however restoring an openvz template does not work. If I remember correctly, sshfs and links do wot get along. here is some of the error message at pve :

Code:

Creating container private area (/mnt/pve/fbc1-nfs/template/cache/debian-7.0-FBC_7.0_amd64.tar.gz)
tar: ./lib/libnl-3.so.200: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/terminfo/r/rxvt-m: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/libxtables.so.7: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libnsl.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/librt.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libuuid.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libudev.so.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libpng12.so.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libncurses.so.5: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libpamc.so.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libbz2.so.1.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libbz2.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
...
tar: ./usr/lib/pymodules/python2.7/SOAPpy-0.12.0.egg-info: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./usr/lib/pymodules/python2.7/python_debianbts-1.10.egg-info: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/SOURCES.txt: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/top_level.txt: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/PKG-INFO: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/dependency_links.txt: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./usr/lib/pymodules/python2.7/debianbts.py: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./usr/lib/python2.7/sitecustomize.py: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib64/ld-linux-x86-64.so.2: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: Exiting with failure status due to previous errors
vps-create ERROR: Unpack /mnt/pve/fbc1-nfs/template/cache/debian-7.0-FBC_7.0_amd64.tar.gz failed
Creation of container private area failed 

TASK ERROR: command 'vzctl --skiplock create 8010 --ostemplate /mnt/pve/fbc1-nfs/template/cache/debian-7.0-FBC_7.0_amd64.tar.gz --private /fbc246-data/private/8010' failed: exit code 48

I think the error lines are all due to symlinks ...

Next I'll try a kvm on sshfs.

**bread-baker** · 05-30-2012

KVM restore works.

I'll try migration next.

**bread-baker** · 05-30-2012

Code:

May 30 12:59:58 starting migration of VM 8020 to node 'fbc240' (10.100.100.240)
May 30 12:59:58 copying disk images
May 30 12:59:58 starting VM 8020 on remote node 'fbc240'
May 30 12:59:59 starting migration tunnel
May 30 13:00:00 starting online/live migration on port 60000
May 30 13:00:02 migration status: completed
May 30 13:00:02 migration speed: 256.00 MB/s
May 30 13:00:04 migration finished successfuly (duration 00:00:06)
TASK OK

so on line migration works . but I'll use the kvm a while and check for problems.

**bread-baker** · 05-30-2012

I had too many errors with rsync and the kvm on sshfs .

the issues may have had nothing to do with sshfs, b ut the system i restored works without issues on drbd and local storage.

So I give up on sshfs for now. Will use nfs instead.

here were some of the issues:

Code:

[mntent]: warning: no final newline at the end of /etc/mtab                                                                                    
[mntent]: warning: no final newline at the end of /etc/mtab                                                                                    
[mntent]: warning: no final newline at the end of /etc/mtab


inflate returned -3 (0 bytes)                                                                                                                  
rsync error: error in rsync protocol data stream (code 12) at token.c(547) [receiver=3.0.3]                                                    
rsync: connection unexpectedly closed (10290 bytes received so far) [generator]                                                                
rsync error: error in rsync protocol data stream (code 12) at io.c(635) [generator=3.0.3]     


rsync: readlink "/bkup/u/hadjx1.pro" failed: Input/output error (5)                                                                            
rsync: recv_generator: failed to stat "/bkup/u/hadjx1.pro": Input/output error (5)                                                             
rsync error: some files could not be transferred (code 23) at main.c(1524) [generator=3.0.3]

**e100** · 05-31-2012

Using sshfs for VM storage is certainly an intriguing idea.

The main issue I see is that SSH uses a lot of CPU unless you have hardware accelerated crypto.
Many CPU's have AES-NI acceleration but squeeze does not have AES-NI enabled for SSH.

The fastest cipher, without hardware acceleration, is arcfour.
If you are using sshfs on a trusted network using arcfour cipher for the sshfs connection should help with performance and reduce your CPU usage.

Just pass the option when mounting sshfs:

Code:

-o Ciphers=arcfour

**RobFantini** · 05-31-2012

hi there e100

i have this in our global ssh_config. do you know if arcfour weaknesses are fixed? In any case It would be great id sshfs issue could get fixed. I've an extra server that I can use for testing if any one has suggestions for retesting sshfs.

Code:

# Changing the ciphers used by SSH to less cpu-demanding ones can improve speed. In this aspect, the best choices are arcfour and blowfish-cbc. Please do not do this unless you know what you are doing; arcfour has a number of known weaknesses. To use them, run SSH with the "c" flag, like this: 
# 2012-04-21 put blowfish-cbc 1-st as 'arcfour has a number of known weaknesses.'
Ciphers blowfish-cbc,arcfour