Would it work to explicitly call the data in the geom_sf()? Since you are explicitly calling in geom_text(), you also need to call it in geom_sf()

drought_sf %>%
  left_join(drought_rmse, by = 'GEOID') 

  ggplot() +
  geom_sf(data = drought_sf, 
          mapping = aes(fill = estimate), 
          color = NA, alpha = 0.8) +
  labs(fill = "RMSE") +
  scale_fill_viridis_c(labels = scales::dollar_format()) +
  geom_text(color = 'black', data = cities_data, check_overlap = TRUE, aes(x = long, y = lat, label = fixed_name))