There wasn’t really any challenge to it because of the absolute nature of the states control and the concept of human rights not really being a thing at that point under feudalism. The most powerful oppositional forces to the feudal lords were mainly the clergy originally, although they had many shared interests, and eventually the mercent class who was cosmopolitan and traveled.

Under feudalism around 20% of the population was considered nobility and the rest were peasents, serfs, or coloni. Nobles generally owned all land and businesses and sometimes had political rights or fiefdoms which were like property that could be passed down.

Midieval societies often used torture, execution, and abuse as well as religious brainwashing to control their subjects. The nobility also guarded knowledge about war and currency and other things, like reading to oppress the subjects who worked the land.

Taxes were generally low. Around 20-25% of their crop. The rights of serfs varied from region to region.

Really the biggest adversary to feudalism was other feudal lords, then the clergy, and at times conspiracies of finance and empires. The feudal kingdoms often aided each other in giving asylum to war criminals or relieving each other from riots. They often intervened to put back into place other monarchies when they were overthrown.