Based on @stefan's suggestion, I binned the real-time stamp variable. As they mentioned, there is a lot of variation in the Buffer variable and real-time stamp.
Here is the code I used:
ggplot(DT, aes(x = Real_Time_Stamp, y = Buffer)) +
geom_line(aes(color = FVN, group = FVN), stat = "summary") +
scale_x_binned(name = "\nTime (s)",n.breaks = 100, limits = c(0,20), breaks = seq(0,20, by = 1))+
scale_y_continuous(name = "\nBuffer Values", limits = c(-0.5,2.5),breaks = seq(0,2, by = 1)))
And here is the plot I got: