Proxmox 4.4 failed - [database] crit: missing directory inode

evgeniy2626

New Member
Feb 3, 2017
2
0
1
36
Hi there, please help

One day I wanted to change hosname of PROXMOX node and to create cluster with two nodes.

I just changed hostname in /etc/hosts /etc/resolv.conf

After that in the webGUI I see two nodes with two names,

and I read that needs to cp -r /etc/pve/nodes/node1/* /etc/pve/nodes/node-new

ADN I DID IT.

Then I get error like as ;
pve-cluster[database] crit: found entry with duplicate name (inode = 0000000000000160, parent = 00000000000000F2, name = 'qemu-server')
[database] crit: DB load failed
[main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'

following the instruction http://blog.sjas.de/posts/proxmox-unable-to-open-database.html I was doing step by step as decribed. I deleted few records from db which associated with new node name (also I changed hostname on old hostname)

And this is does not helped me.
I still can't run pve-manages and others ....
And after that the directory /etc/pve/ is empty, how can I restore the node with all vm's?

# systemctl status pve-cluster.service
● pve-cluster.service - The Proxmox VE cluster filesystem
Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled)
Active: failed (Result: exit-code) since Fri 2017-02-03 20:05:21 NL; 1min 31s ago
Process: 5336 ExecStart=/usr/bin/pmxcfs $DAEMON_OPTS (code=exited, status=255)

Feb 03 22:05:21 base pmxcfs[5336]: [database] crit: missing directory inode (inode = 00000000000C9C1F)
Feb 03 22:05:21 base pmxcfs[5336]: [database] crit: DB load failed
Feb 03 22:05:21 base pmxcfs[5336]: [database] crit: missing directory inode (inode = 00000000000C9C1F)
Feb 03 22:05:21 base pmxcfs[5336]: [database] crit: DB load failed
Feb 03 22:05:21 base pmxcfs[5336]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Feb 03 22:05:21 base pmxcfs[5336]: [main] notice: exit proxmox configuration filesystem (-1)
Feb 03 22:05:21 base systemd[1]: pve-cluster.service: control process exited, code=exited status=255
Feb 03 22:05:21 base systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Feb 03 22:05:21 base systemd[1]: Unit pve-cluster.service entered failed state.



# pmxcfs -l
[database] crit: missing directory inode (inode = 00000000000C9C1F)
[database] crit: DB load failed
[main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
[main] notice: exit proxmox configuration filesystem (-1)



# journalctl -xn
-- Logs begin at Fri 2017-02-03 21:35:49 NL, end at Fri 2017-02-03 22:48:39 MSK. --
Feb 03 22:48:39 base pveproxy[2595]: worker 8863 finished
Feb 03 22:48:39 base pveproxy[2595]: starting 1 worker(s)
Feb 03 22:48:39 base pveproxy[2595]: worker 8867 started
Feb 03 22:48:39 base pveproxy[8867]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/HTTPServer.pm line 1646.
Feb 03 22:48:39 base pve-ha-lrm[2601]: ipcc_send_rec failed: Connection refused
Feb 03 22:48:39 base pve-ha-lrm[2601]: ipcc_send_rec failed: Connection refused
Feb 03 22:48:39 base pve-ha-lrm[2601]: ipcc_send_rec failed: Connection refused
Feb 03 22:48:39 base pve-ha-crm[2590]: ipcc_send_rec failed: Connection refused

# /etc/init.d/pve-manager status
● pve-manager.service - PVE VM Manager
Loaded: loaded (/lib/systemd/system/pve-manager.service; enabled)
Active: failed (Result: signal) since Fri 2017-02-03 23:34:42 MSK; 25s ago
Process: 11520 ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall (code=killed, signal=KILL)
Main PID: 11520 (code=killed, signal=KILL)

Feb 03 23:21:58 base pve-manager[11520]: <root@pam> starting task UPID:base:00002D4:0009BA54:5894E666:startall::root@pam:
Feb 03 23:21:58 base pve-manager[11520]: writing cluster log failed: ipcc_send_rec failed: Connection refused
Feb 03 23:34:42 base systemd[1]: pve-manager.service: main process exited, code=killed, status=9/KILL
Feb 03 23:34:42 base systemd[1]: Failed to start PVE VM Manager.
Feb 03 23:34:42 base systemd[1]: Unit pve-manager.service entered failed state.
Feb 03 23:35:05 base systemd[1]: Stopped PVE VM Manager.

I noticed /dev/fuse is not mounted

~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 10M 0 10M 0% /dev
tmpfs 6.3G 25M 6.3G 1% /run
/dev/dm-0 950G 90.5G 81G 11% /
tmpfs 16G 0 16G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 16G 0 16G 0% /sys/fs/cgroup

Please help me!
 
Last edited:
I do not know the linked third party howto, but hacking into the sqlite database is for sure not the way.

If you cannot fix it, better re-install or consult an Proxmox VE expert.
 
I do not know the linked third party howto, but hacking into the sqlite database is for sure not the way.

If you cannot fix it, better re-install or consult an Proxmox VE expert.
yes! its true

The only one way will help us)) as me too
 
I thought I'd post here because this was the only post that came up after googling what when wrong my sqllite database after changing my hostname. I was getting this error that my inode was missing.

Upon opening the SQLlite database the inode that was missing when converted from hex to decimal matched the parent of my `qemu-server` entry. Looking at the directory structure, this was supposed to be the new hostname. Because sqllite is supposed to be used to create a directory structure of /etc/pve/myhostname/qemu-server. With the myhostname inode missing, I needed to create it.


My missing inode was 00000000000188CB which hex->decimal converts to 100555.

I ran this command to find out it was my qemu-server.
Code:
select * from tree where parent = 100555

I ran this sql command on my database and uploaded it to the server (type = 4 is a folder).
Code:
insert into tree (inode, version, parent, type, writer, name, mtime) values (100555, 100555, 0, 4, 0, 'myhostname', 1619669401)

I started the cluster twice with
Code:
service pve-cluster start

I could then login to my cluster

To fix the errors with no stats I also had to run
Code:
systemctl start pvestatd

I hope this helps someone, killed a few hours of my time.
 
Last edited:
I thought I'd post here because this was the only post that came up after googling what when wrong my sqllite database after changing my hostname. I was getting this error that my inode was missing.

Upon opening the SQLlite database the inode that was missing when converted from hex to decimal matched the parent of my `qemu-server` entry. Looking at the directory structure, this was supposed to be the new hostname. Because sqllite is supposed to be used to create a directory structure of /etc/pve/myhostname/qemu-server. With the myhostname inode missing, I needed to create it.


My missing inode was 00000000000188CB which hex->decimal converts to 100555.

I ran this command to find out it was my qemu-server.
Code:
select * from tree where parent = 100555

I ran this sql command on my database and uploaded it to the server (type = 4 is a folder).
Code:
insert into tree (inode, version, parent, type, writer, name, mtime) values (100555, 100555, 0, 4, 0, 'myhostname', 1619669401)

I started the cluster twice with
Code:
service pve-cluster start

I could then login to my cluster

To fix the errors with no stats I also had to run
Code:
systemctl start pvestatd

I hope this helps someone, killed a few hours of my time.
Saved me, thank u so much!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!