Answering my own question to help others who might face the same issue in the future. While I still don't understand why in some cases the script_score gets called more than once, I was able to fix the scoring.
To prevent the scoring from being summed or multiplied I added boost_mode: replace parameter like below:
{
"query": {
"function_score": {
"query": { ... },
"boost_mode": "replace", // Adding this fixed the issue for me
}
}
I found this solution by looking at OpenSearch docs https://opensearch.org/docs/latest/query-dsl/compound/function-score
You can specify how the score computed using all functions[1] is combined with the query score in the
boost_modeparameter, which takes one of the following values:
multiply: (Default) Multiply the query score by the function score.replace: Ignore the query score and use the function score.sum: Add the query score and the function score.avg: Average the query score and the function score.max: Take the greater of the query score and the function score.min: Take the lesser of the query score and the function score.
[1] Note that the boost_mode works in both scenarios: whether you have a single function (as in my case) or multiple functions (also in case of multiple functions you might want to look at score_mode parameter too from the same docs page that I provided its link above)