Thanks to @vladimir-botka's hint mentioning the template lookup plugin is a string, I figured an easy fix:

storage:
  accessModes:
    {{ modes | to_nice_yaml | trim | indent(4) }}
  resources:
    requests:
      storage: 5Gi

The trim takes care of the problematic code output:

ok: [localhost] =>
  msg: |-
    storage:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 50Gi