Not sure I understand your use case, but if the problem is that you get duplicates in your aggregation, maybe you can try to use distinct on your string aggregations:

https://docs.djangoproject.com/fr/5.1/ref/contrib/postgres/aggregates/#stringagg