Remove nodesToDelete functionality

[stuggi]

As the nodesToDelete does not describe an end result of the CR,
this will be removed with this patch. To remove a specific node
or nodes the following process needs to be followed:

1. annotate the machine/machines object corresponding to the node
   which should be removed
2. lower the worker number of the CR of the machineset

[stuggi]

We might want to:

• not delete any node before it gets annotated
• remove the workers parameter from the CRD and
  • use the machineset to scale
  • watch machines for annotation and then start removal

[dprince]

lgtm on this commit. One related topic we might discuss is only deleting nodes that have been annotated. So essentially we'd force users to annotate any nodes created via this machineset in order to delete them as a safety feature.

[stuggi]

@luis5tb what you think about this and my above comment?

[luis5tb]

@luis5tb what you think about this and my above comment?

Umm, I have mix feelings. On the one hand it is not a clear fit for the CRD, but on the other hand, the idea is that the user only need to know about the CRD, not about the machineset and even less about the machine objects. And, to some extend, the CRD spec and status describe the end result. The nodesToDelete on the spec describe what nodes should be tagged with that annotation, and the status which ones are actually tagged already. So it is similar to other config options I think.

Is there a reason to remove it?

[stuggi]

@luis5tb what you think about this and my above comment?

Umm, I have mix feelings. On the one hand it is not a clear fit for the CRD, but on the other hand, the idea is that the user only need to know about the CRD, not about the machineset and even less about the machine objects. And, to some extend, the CRD spec and status describe the end result. The nodesToDelete on the spec describe what nodes should be tagged with that annotation, and the status which ones are actually tagged already. So it is similar to other config options I think.

Is there a reason to remove it?

The issue I see with the nodesToDelete is that as soon the delete is finished, the information in the CR is out of date.
Someone would need to edit the CR and clean the content of the nodesToDelete section as we don't want the operator
to edit its own cr.

Therefore in one of our CRD meetings (when you were on PTO) the result of discussion was that it might be better
to not use the CRD to manage the nodesToDelete and the user would need to annotate them before scaling down.

With this I thought about this again while creating this patch on if we should use the CR for scaling at all. Might it be
better/more native OCP like when we use the CRD just for configuration and for scaling using the machineset direct,
but the main thing is someone have to cleanup nodesToDelete section after a scale down which can be/will be confusing for
an end user.

[stuggi]

rebased after sdk upgrade