It is possible by triggering a dynamic update to some of the ssl related configurations. See here. But not sure if your old apache kafka is having this feature already.

Maybe in future the trigger is also not needed anymore: see KIP-687

Details this answer