I found really ugly solution. Here i posted an answer about doing the same, but with the default TextArea

So I create invisible responsive TextArea like in that answer. Its existence is only to obtain a desired height of the message. Then we save height, delete it from parent node and replace it with RichTextArea with calculated height.

    heightCalculation = new CompletableFuture<>();
    TextArea textArea = new TextArea();
    textArea.getStyleClass().add("message-text");
    textArea.setEditable(false);
    textArea.setWrapText(true);
    textArea.setVisible(false);
    messageContentContainer.getChildren().add(textArea);
    setupTextAreaDimensionsChangeEventListener(textArea);
    textArea.setText(message);
    heightCalculation.thenAccept(calculatedHeight -> {
        messageContentContainer.getChildren().remove(textArea);
        Document document = new Document(message);
        RichTextArea richTextArea = new RichTextArea();
        richTextArea.setEditable(false);
        richTextArea.setMinHeight(calculatedHeight);
        richTextArea.setPrefHeight(calculatedHeight);
        richTextArea.setMaxHeight(calculatedHeight);
        richTextArea.getActionFactory().open(document).execute(new ActionEvent());
        richTextArea.getActionFactory().save().execute(new ActionEvent());
        messageContentContainer.getChildren().add(richTextArea);
    });

Message bubble, area between grey horizontal separators is a RichTextArea