I got it, for future reference:

p <- ggplot(df, aes(x = Year, y = Average, color = code)) +
  geom_point(aes(size=3, alpha=0.8,group = seq_along(Year))) +  # Adiciona os pontos
  geom_line(aes(y = smooth_values), size = 1) +  # Adiciona as linhas de tendência
  labs(title = "Evolução do Average por Região",
       x = "Ano", y = "Average") +
  theme_minimal() +
  theme(legend.title = element_blank())

The trick was on seq_along(Year) as shown [here][1] on stackoverflow...