Having lot of issues with new nodes

[Red Squirrel]

I was playing around with Proxmox by virtualizing 2 nodes on my existing ESXi infrastructure, since it was taking forever for my new hardware to come through the mail and I was getting annoyed I started migrating VMs over anyway by converting the vmdks and creating new VMs. Things were going well, with live migrates etc too as I was testing stuff.

My hardware finally showed up. I stuck a 4 port nic in each one, for now I just need 2 ports, so using the built on one for management (and storage traffic for now) and the other port on the 4 port nic is a trunk port to the switch.

1st problem:

vlans are not working on the new nodes, it's set as a trunk, the vlans are allowed, all that stuff should be fine as far as the switch is concerned, I set vlan aware to yes when I created the bridge, and the name matches the other nodes. But minute I migrate a VM to it, there's no network. What further troubleshooting or logs could I look at to figure out what's going on?


2nd problem: one of the nodes is showing offline, no matter what I do. I can open a console and even migrate a VM to it, but there's always an X icon. Sometimes if I click on it, I get prompted for a password then get an "invalid tivcket" error and get kicked out of the entire cluster. At one point it even looked like the whole cluster was corrupted and as I was typing this, it magically started to work again. And now as I type this the node that was showing offline is now showing online, but the one that was showing online is now showing offline. (both new physical nodes). There is no network interfaces showing up at all either on the node that was previously showing offline and now magically is showing online.

Every now I keep getting kicked out of the entire cluster and need to login again and everything is just super unstable, like it won't show anything or just sits spinning etc. it seems adding these two physical nodes to the cluster completely broke everything.

Overall there seems to be lot of weird instability stuff going on right now. What would be the best way to even start troubleshooting this?

 

[Red Squirrel]

Yeah I think the whole cluster just got corrupted somehow. Now I don't even see anything. Just the datacentre node. How did this happen and is there any way to recover this?


Edit: Shutdown both new nodes. Then everything started to work again. Decided to turn one node up only to see what happens. Then it seemed to be working ok, took this chance to troubleshoot networking further while the GUI was not acting weird. Turns out "active" doesn't mean what I thought it did. I figured it meant it had a link, so I thought I got lucky and picked the right one the first time and left it at that but turns out I had assigned the wrong physical nic to the vmbr1. So after figuring that out vlans work now and I can migrate VMs and they work.

Turned up the 2nd node and did the same with the networking and now everything magically works...

No idea what this was about. I suppose the bad network config could have been an issue but I can't see why as no actual networking would have been happening on those ports. The lights were off, which is what got me to investigate further.

Going to go to bed and keep playing with this tomorrow to see if it stays stable. It was completely crashing and kicking me out and doing all sorts of weird things minutes ago, and reboots were not doing anything. Only shutting down the nodes completely then turning them back up one by one seemed to helped.