I am having same issue (sorry I can't comment just post "answer")
so what you are saying is that you were missing to join the worker nodes to the raft cluster, therefore you couldn't authenticate using kubernetes method?
I just followed your steps to join the workers to raft cluster but still I am unable to authenticate pfff