The score will increase by the highest point that you achieve per opponent and not per battle.
When you battle with the same player in multiple times, the final score that will be added is only from the highest score that you achieve from that player.