Duplicates can't be removed fully, but there are news api's that provide fields that make deduplication easier on the client side. Like Newsdata.io,

it provides each article with a unique article_id, article link, source_id, source_url, source_icon, enabling developers to filter repeat on their end. It also provides a "removeduplicate" parameter which allows users to filter out duplicate articles.

Additionally, It allows filtering by category, language, country, Region etc, which can help narrow results and minimize duplicate headlines at the API level before you process them.