Thx to @shingo: Using TypeNameHandling = TypeNameHandling.Auto (TypeNameHandling.All is also possible) solved my issue. And there is no need to write custom converter at all.
TypeNameHandling = TypeNameHandling.Auto
TypeNameHandling.All