In the description text on the Final Score, "every player you remain undefeated by" actually means "every player your DEF Party remains undefeated by",
so your Score will not be affected in any way by losing a battle you challenged another player to.
Customer Support