Accidentally took down the panel

[ciarandwi]

I'm just a developer who has no experience of Networking.

We had some issues with a server (Server 4) and it would quite frequently go down. The server was moved but then Server 2 couldn't communicate with Server 4 anymore.

After doing some looking into it and talking to another person who's quite smart and has his own business, he was trying to help me debug the situation and sent me a command of 'ifdown vmbr1' 'ifup vmbr1' to be ran on Server 4 however, I accidentally ran this on Server 2 and now Server 2 is not SSHable nor the panel link doesn't work.

From my understanding, ifup sets the default interface so the 8006 panel link doesn't work.

I can boot the server into rescue mode but i'm a little confused as to what I can do get my containers back.

 

[waltar]

ifdown/-up are just runtime cmd's and don't do config permanent changes. If you don't have TFT,keyboard+mouse at server just reboot them both first when it's comfortable for your running vm's etc for now.

 

[ciarandwi]

ifdown/-up are just runtime cmd's and don't do config permanent changes. If you don't have TFT,keyboard+mouse at server just reboot them both first when it's comfortable for your running vm's etc for now.

This still wouldn't explain the fact the :8006 url would be down nor it not being SSHable. It just happened as soon as I ran the command and then all of our websites went down with a Cloudflare error page.

I've rebooted the server in normal mode and the page :8006 page still isn't up

I've been talking to Scaleway (server people) and they can't figure it out. I'm about to go on a call with somebody who knows a lot more about systems than I do so i'm hoping they they can resolve this issue.

I appreciate your input

 

[dakralex]

ifup and ifdown (de)configure network interfaces, which are defined in /etc/network/interfaces) only temporarily. If vmbr1 is the network interface which is used to hold your ssh connection, then it was cut as soon as it was deactivated by the ifdown vmbr1 command. It's advisable to do changes to the network with physical access to a connected terminal only or run those commands in such a way that they will be run even if your connection is cut (e.g. ifdown vmbr1; ifup vmbr1 in a screen session).

 

[dakralex]

I've rebooted the server in normal mode and the page :8006 page still isn't up

Do you have physical access to the server itself? It would also be helpful to know more about the network configuration as it seems that they were misconfigured in the first place when Server 2 could not reach Server 4.

 

[ciarandwi]

ifup and ifdown (de)configure network interfaces, which are defined in /etc/network/interfaces) only temporarily. If vmbr1 is the network interface which is used to hold your ssh connection, then it was cut as soon as it was deactivated by the ifdown vmbr1 command. It's advisable to do changes to the network with physical access to a connected terminal only or run those commands in such a way that they will be run even if your connection is cut (e.g. ifdown vmbr1; ifup vmbr1 in a screen session).

I don't utilise screen enough and that's something I need to learn and I now wish I did all this in that environment so it wouldn't have broken everything

 

[waltar]

Normally first interface is together with vmbr0. Can you ping your servers (2+4) now ?

 

[ciarandwi]

Normally first interface is together with vmbr0. Can you ping your servers (2+4) now ?

Pinging the public IP will return with a Request Timeout

Luckily, we have another server where I can see Server 2. When I go to the shell on there, it says 'ssh: connect to host IP port 22: No route to host' so at least now I have a different error that I can try to solve.

I can work with errors but working with something that is unknown to me is harder.

 

[waltar]

That's bad (ssh is port 22 but that network is down at server2), do you have another internal network address to that server also ? Otherwise you need to go directly in front of it.

 

[ciarandwi]

Issue has been resolved due to some help of a senior

The Issue was that all the interfaces went down on our one of the servers resulting in it not seeing the panel link. I submitted ticked to our Server providers and not once did they mention about using KVM (I learnt about what this actually was).

We had to go onto the machine using KVM and then reset all the interfaces back up resulting in all the servers getting booted back up

Thanks for the support and suggestions. Networking and servers are a new domain to me so it's all one big learning lesson for myself.

Just thought i'd update you all