My flatmate gave her boyfriend a key. How do I raise the change to our household?

I came home early and found my flatmate's boyfriend making lunch. He held up the spare key and said, "She said it was easier than waiting outside." I hadn't known there was a spare key in circulation.

He first stayed because the heating at his place was being repaired. We talked about two weeks. Six weeks later, he is here three or four nights a week, and sometimes during the day when she's out. He washes up, buys some groceries and has always been polite to me. The trouble is that I chose a home with one flatmate, and now I check who's in the kitchen before coming out in my dressing gown.

When I mentioned the key, she said I was making him feel unwelcome after he'd helped us carry a new sofa upstairs. I can appreciate the sofa and still feel surprised by finding him home alone. I haven't asked him to leave or changed any locks. Our tenancy terms also need checking before we talk about a permanent arrangement; I don't know what they allow.

How do I ask for an agreed end to the temporary stay and a conversation about the key without turning every ordinary visit into a negotiation?