I had the same issue, struggled to remove the colors but in the end found this in the docs:
pipeline_stages: - decolorize:
https://grafana.com/docs/loki/latest/send-data/promtail/stages/decolorize/