I don't believe that all interactions give others a hold over me -- only when I unilaterally ask for something personal, or else when my interaction requires some inconvenience to the other person(s)... stopping conversation momentarily, or interrupting a shared activity ... in order to include me. Or my presence discomfits someone that is already there.

It isn't that I feel burdened, myself... more than I worry about burdening others, thereby incurring obligation and losing some measure of future control over my actions.