Major Problem Host node will not start

[jhamon]

We have plugged a monitor in and the server tries to perform a fsck but fails with the error:

No such file or directory while trying to open /dev/pve/root

Mount point /dev/shm does not exsist. Skipping mount.

<file system. <mount point> <type> <options> <dump> <pass>
dev/pve/root / ext3 errors=remount-ro 0 1
dev/pve/data /var/lib/vz ext3 defaults 0 1
dev/sda1 /boot ext3 defaults 0 1
dev/pve/swap none swap 0 0
proc /proc proc defaults 0 0
/etc/init.d/networking start
configuring network interfaces... ifup: failed to open statefile /etc/network/run/ifstate: read-only file system

How do I fix this?

 

[tom]

We have plugged a monitor in and the server tries to perform a fsck but fails with the error:

No such file or directory while trying to open /dev/pve/root

Mount point /dev/shm does not exsist. Skipping mount.

<file system. <mount point> <type> <options> <dump> <pass>
dev/pve/root / ext3 errors=remount-ro 0 1
dev/pve/data /var/lib/vz ext3 defaults 0 1
dev/sda1 /boot ext3 defaults 0 1
dev/pve/swap none swap 0 0
proc /proc proc defaults 0 0
/etc/init.d/networking start
configuring network interfaces... ifup: failed to open statefile /etc/network/run/ifstate: read-only file system

How do I fix this?

what did you exactly? Hardware failure (Disks)? Do you have a Raid?

 

[dietmar]

What version do you use? Is this a new install or an existing server? What happened at all - please can you give us some details?

- Dietmar

 

[jhamon]

Hi,

version is 0.9 installed around 4 months ago. Updated to 0.9 B2 when that was released.

A couple of the VPSs stopped working and would not restart. When host machine was rebooted failed to restart.

Using Raid 1 via 2 x 500 Gb SATAII hard disks with a 3WARE 9650SE controller.



Jonathan Hamon

 

[tom]

Hi,

version is 0.9 installed around 4 months ago. Updated to 0.9 B2 when that was released.

A couple of the VPSs stopped working and would not restart. When host machine was rebooted failed to restart.

Using Raid 1 via 2 x 500 Gb SATAII hard disks with a 3WARE 9650SE controller.

Jonathan Hamon

Check your hardware raid for errors, disk smart status. Do you have a backup of your virtual machines (with vzdump)? if yes, it would be the fastest way to reinstall a clean Proxmox VE 1.0 and restore the VM´s.

Before you start make sure the hardware is ok.

 

[jhamon]

Hi,

I have backup of VPS nodes but these are stored on the hard disk of this server i /var/lib/vz/dump I believe and due to the errors as outlined I don't seem to be able to get access to these to copy to another location.

We have tried booting the server with a CD but the problem is knoppix centos and fedora do not seem to recognise the network card.

Is there a bootable recovery CD or USB drive I can use with Debian 4?


Jonathan Hamon

 

[mneuger]

Is there a bootable recovery CD or USB drive I can use with Debian 4?

You can try an ubuntu live cd - they have quite good hardware support. Or you use the proxmox install cd (and aborting installation).

 

[tom]

HowTo: Use Proxmox VE debug mode to access data on unbootable systems

This is a small howto describing the steps to start from Proxmox VE CD in order to have network access and access to the data stored on the local disks. (e.g. can be used if the /root partition is dead because of a hardware failure)

1. Make sure you have a running DHCP server
2. Boot your system from Proxmox VE CD-Rom
3. On the start prompt, type "debug"
4. On the first prompt, type "CTRL-D" to continue
5. On the page showing the Proxmox license, click "Abort" to go back to the console
6. Start LVM:

   /etc/init.d/lvm start

7. Mount the data partition:

   mount /dev/pve/data /mnt

8. Now you have access to your data partition on /mnt.
9. As you got a running network auto configured via DHCP you can scp your data to another place

HowTo: Boot with the Kernel on the Proxmox VE CD-Rom:

1. Boot your system from Proxmox VE 1.0 CD-Rom
2. On the start prompt, type "pveboot"
3. Your system boots up using the Kernel on the CD

And do not forget to buy reliable hardware with a good raid controller ...

 

[Erwin123]

And do not forget to buy reliable hardware with a good raid controller ...

What do you consider a good raid controller? Are there bad ones?

 

[mneuger]

What do you consider a good raid controller??

A good controller is one you are familiar with, and you know that it is working in case of errors ;-)

Are there bad ones?

yes. If you loose data on the disk you got a bad one.

 

[tom]

What do you consider a good raid controller? Are there bad ones?

Yes, there are a lot of them. here are some points making a raid controller "good" for me - all others missing one of these points are not optimal.

• Support of all Linux Distro´s by the default Kernel
• Management tools working on all Linux Distro, offering remote management, email alarms, etc. (e.g. Adaptec ASM)
• Audio Alarms
• Support of hot swap, including cage support with lights
• Batteries backup (cheap controllers does not have this option)
• Fast processor, minimum 128 ram
• 3 years warranty, quit replacements

This list is not complete but will help to get started. If you send this list to your hardware vendor he should be in the position to choose the right one.

 

[ProxmoxUser]

I have a similar problem and have booted from CD as you describe, how can I copy the files to another server over the network as ftp/ssh/scp/rsync arent available..

 

[jhamon]

Hi,

seems ftp scp ssh etc are NOT available so do we need to install this first with a command line something like this

apt-get install scp ftp ssh or if not correct please give further information



Jonathan Hamon

 

[jhamon]

Also cannot install scp due to CD being a read only media. Can proxmox please fix this quick so that scp ftp and ssh are installed on CD so we can do what you suggest.



Jonathan Hamon

 

[dietmar]

Just uploaded a new CD image. We now include ssh/scp and ftp

- Dietmar

 

[jhamon]

Thanks


Jonathan Hamon