How do I stay out of the office lottery pool without spending every draw worrying they'll win?

The office pool organiser waved the ticket sheet and said, "Last chance for this week. You don't want to be the only one still coming in on Monday."

It's $5 a week. I haven't joined because I don't want lottery spending in my budget. Still, when colleagues start imagining what they'd do with a win, I picture being the person who saved five dollars and missed everything.

I can afford an occasional $5. The trouble is that 'occasional' doesn't seem to describe the invitation: it comes every week, and I've already declined four times. Buying in would make the awkward moment disappear, but I suspect I'd then feel the same pressure to keep joining.

Nobody has threatened to exclude me from lunch or normal work. It's the shared daydream I feel outside of. Would you give one firm answer and stop discussing the draws, ask the organiser to leave you off the reminders, or consider the weekly ticket an entertainment cost? I want the decision to belong to my budget rather than to the joke about Monday.