So I was able to resolve / workaround this issue.
With a bit force I was able to remove the 2 affected nodes out of the cluster and reinstall them. Everything works like a charm now.
One point I would like to be changed in future releases are the logmessages, because they are quite confusing.
In the admin interface the log entry tells me that one node, let’s call it node1, was not able to connect to the master node.
But in the indexer log file of all other nodes in the cluster I see that this particularly node1 was marked as failed node.
From my point of view the log message in the admin interface should display which node sees which nodes as failed. This would generate more log entries but would help a lot to find out which nodes are misbehaving.
The reason for this change is that I was not 100% sure if all the other nodes are not working properly or only node1.
Thanks to @deepkaran.salooja for the help