We use ncipollo/release-action@v1 GitHub action to make releases. I don't why it was working well before, but anyway I found I can pass makeLatest: true for the release and it will be marked as latest. It resolved the problem.
ncipollo/release-action@v1
makeLatest: true
latest