I was able to get my use case to work, following these examples and tips from June Choe and Cara Thompson:

• https://yjunechoe.github.io/posts/2021-06-24-setting-up-and-debugging-custom-fonts/
• https://www.cararthompson.com/posts/2024-01-12-using-fonts-in-r-for-dataviz/2024-01-12_getting-fonts-to-work#check-the-font-you-want-to-use-is-installed

Using a combination of systemfonts::register_font() for the TTF I wanted to register and systemfonts::registry_fonts() to confirm it's existence, I could then load the font without impacting emoji rendering. Even can combine the two in the same plot:

df <- cars |>
  mutate(
    glyph = case_when(speed > 16 ~ "✅",
                      .default = fontawesome("fa-car")),
    label = case_when(speed > 16 ~ "Fast",
                      .default = "Slow"),
    color = case_when(speed > 16 ~ "red",
                      .default = "green")
  ) |>
  tibble::tibble()

ggplot() +
  geom_swim_marker(
    data = df,
    aes(x = speed, y = dist, marker = label),
    size = 5, family = "FontAwesome"
  ) +
  scale_marker_discrete(glyphs = df$glyph,
                        colours = df$color,
                        limits = df$label)