This question reminds me of Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure".

Your model minimizes the error based on the loss function. Common loss functions like Mean Squared Error have well-studied behavior and performance. I would recommend sticking with those, as optimizing for a metric may elicit unwanted model behavior.

As an example, I once used R-squared as a loss function for a time-series regression task instead of Mean Squared Error. As a result, the model's predictions almost completely ignored outliers (or overfit on them, I don't remember exactly) in the dataset, which was not optimal for my task. Returning to Mean Squared Error yielded better results. Perhaps it may be the same with yours.