The All feed is for All. If you don’t want All don’t use All.

It’s designed as a White List system, with subscriptions. You pick the things you want to see. You’re asking for a Black List system, where you see everything accept what you don’t want. That only works while things are small. As they grow, Black Lists become much larger, and more difficult to manage. When you get to the point where their are thousands of communities with tens of thousands of posts every second, they become useless.