sshfs shared storage

B

bread-baker

Member
Mar 6, 2010
432
0
16
I think it would be good to allow sshfs for shared storage.

For us sshfs is more reliable then nfs . We use sshfs for user documents - with $HOME/Documents targeted to a kvm sshfs . We've done that for 3 years and have had very few issues.
 
never tried but just mount via /etc/fstab and then configure a shared directory storage, does that work for you?
 
the mount and share work, thanks for the idea.

however restoring an openvz template does not work. If I remember correctly, sshfs and links do wot get along. here is some of the error message at pve :
Code: 
Creating container private area (/mnt/pve/fbc1-nfs/template/cache/debian-7.0-FBC_7.0_amd64.tar.gz)
tar: ./lib/libnl-3.so.200: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/terminfo/r/rxvt-m: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/libxtables.so.7: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libnsl.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/librt.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libuuid.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libudev.so.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libpng12.so.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libncurses.so.5: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libpamc.so.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libbz2.so.1.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libbz2.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
...
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/SOAPpy-0.12.0.egg-info: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/python_debianbts-1.10.egg-info: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/SOURCES.txt: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/top_level.txt: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/PKG-INFO: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/dependency_links.txt: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/debianbts.py: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/python2.7/sitecustomize.py: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./lib64/ld-linux-x86-64.so.2: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: Exiting with failure status due to previous errors[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]vps-create ERROR: Unpack /mnt/pve/fbc1-nfs/template/cache/debian-7.0-FBC_7.0_amd64.tar.gz failed[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]Creation of container private area failed[/FONT][/COLOR] 

TASK ERROR: command 'vzctl --skiplock create 8010 --ostemplate /mnt/pve/fbc1-nfs/template/cache/debian-7.0-FBC_7.0_amd64.tar.gz --private /fbc246-data/private/8010' failed: exit code 48

I think the error lines are all due to symlinks ...

Next I'll try a kvm on sshfs.
 
Last edited:
Code: 
[COLOR=#000000][FONT=tahoma]May 30 12:59:58 starting migration of VM 8020 to node 'fbc240' (10.100.100.240)[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 12:59:58 copying disk images[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 12:59:58 starting VM 8020 on remote node 'fbc240'[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 12:59:59 starting migration tunnel[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 13:00:00 starting online/live migration on port 60000[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 13:00:02 migration status: completed[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 13:00:02 migration speed: 256.00 MB/s[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 13:00:04 migration finished successfuly (duration 00:00:06)[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]TASK OK

so on line migration works . but I'll use the kvm a while and check for problems.[/FONT][/COLOR]
 
I had too many errors with rsync and the kvm on sshfs .

the issues may have had nothing to do with sshfs, b ut the system i restored works without issues on drbd and local storage.

So I give up on sshfs for now. Will use nfs instead.

here were some of the issues:
Code: 
[mntent]: warning: no final newline at the end of /etc/mtab                                                                                    
[mntent]: warning: no final newline at the end of /etc/mtab                                                                                    
[mntent]: warning: no final newline at the end of /etc/mtab


inflate returned -3 (0 bytes)                                                                                                                  
rsync error: error in rsync protocol data stream (code 12) at token.c(547) [receiver=3.0.3]                                                    
rsync: connection unexpectedly closed (10290 bytes received so far) [generator]                                                                
rsync error: error in rsync protocol data stream (code 12) at io.c(635) [generator=3.0.3]     


rsync: readlink "/bkup/u/hadjx1.pro" failed: Input/output error (5)                                                                            
rsync: recv_generator: failed to stat "/bkup/u/hadjx1.pro": Input/output error (5)                                                             
rsync error: some files could not be transferred (code 23) at main.c(1524) [generator=3.0.3]
 
Last edited:
Using sshfs for VM storage is certainly an intriguing idea.

The main issue I see is that SSH uses a lot of CPU unless you have hardware accelerated crypto.
Many CPU's have AES-NI acceleration but squeeze does not have AES-NI enabled for SSH.

The fastest cipher, without hardware acceleration, is arcfour.
If you are using sshfs on a trusted network using arcfour cipher for the sshfs connection should help with performance and reduce your CPU usage.

Just pass the option when mounting sshfs:
Code: 
-o Ciphers=arcfour
 
hi there e100

i have this in our global ssh_config. do you know if arcfour weaknesses are fixed? In any case It would be great id sshfs issue could get fixed. I've an extra server that I can use for testing if any one has suggestions for retesting sshfs.
Code: 
# Changing the ciphers used by SSH to less cpu-demanding ones can improve speed. In this aspect, the best choices are arcfour and blowfish-cbc. Please do not do this unless you know what you are doing; arcfour has a number of known weaknesses. To use them, run SSH with the "c" flag, like this: 
# 2012-04-21 put blowfish-cbc 1-st as 'arcfour has a number of known weaknesses.'
Ciphers blowfish-cbc,arcfour
 
Last edited:
You must log in or register to reply here.
Top Bottom