I was able to solve this by inserting a "Word Joiner" unicode character U+2060 in between the inline content and the text. You can try something like this:

val nestedStr = buildAnnotatedString {
    append(prefix)
    append("\u2060")
    appendInlineContent("icon", "?")
}