So your default position on anything like this is 'not included' with the 'slightly negative' connotations that that includes, and the only way to change that is to be specifically invited. Getting that invitation may well be a problem, and might lead to misunderstandings when no such invitation is needed - as in a Pride march.

Really Brian, you're old enough that you should be able to get away from the metaprogramming delivered by a bunch of ignorant jocks in high school - or whoever else induced this behaviour - and you're intelligent enough that you should realise that you're just replaying these games *in the way that the people who victimised you want you to keep replaying them*.

Stop being their victim 35 years on!

[Being blunt because I think you need it, but hugs are available as well].