You are kind of right but your focus is way too narrow and short-sighted. The better off people around me are, the better off I am. Society is what brings me the comforts and freedoms I have today. Without society I would likely be some peasant living in a hut and trying to farm some desolate piece of land until I died at a very early age.
By donating to medical research, I am able to improve medical care for all, and that includes myself. By donating to the arts I get to see art that I would have never been able to see otherwise. By donating to the poor and things like education, I reduce crime and violence where I live.
So yeah, basically everything you do is self-serving, just try not to be such a basic bitch about the scope of your actions.