My squash opponent keeps calling the score while facing away. How do I ask for a change without stopping the match constantly?

I asked, "Was that 4–3 or 5–3?" just as my opponent got ready to serve. My opponent sighed: "We can't stop and go through it after every rally."

I'm hard of hearing, and in that squash court I struggle to follow a score called over someone's shoulder while we both move into position. I told him before we started that I would need him to face me for a moment. He agreed, then slipped back into calling as he walked away.

I don't need a long discussion between points. I want a brief face-to-face score announcement that I can acknowledge. The match is competitive, and I understand that repeated interruptions break the rhythm for him too.

How would you reset the arrangement mid-match, or raise it with the organiser beforehand next time, without making every misunderstanding feel like a demand for special patience? I'd like to spend the evening thinking about the rally rather than guessing the score.