I suggest you split the task into three steps:

1. Create a temporary staging table
2. Copy CSV file/files into staging table
3. INSERT INTO my_table SELECT * FROM staging ON CONFLICT DO NOTHING

This way you will benefit from the batch insertion of COPY and still have to flexibility of SELECT statement