The issue was some of my code chunks that produced plots were set to cache = TRUE. When I removed that argument the xxx_html directory was no longer produced.
cache = TRUE
xxx_html