1. Sometimes breakers don’t trip, so there’s a small risk of fire
2. Restarting the whole house may have large initial loads as everything starts at once: more chance of it happening again or potentially damaging some appliances
3. Risk of heat damage to wiring with repeated trips, risk of broken connections from more frequent expansion from heat/cool cycles
4. Inconvenience, especially in the old days when you’d have to go through to set clocks. If while asleep you might not be awoken in time. If you weren’t home, maybe food gone bad
5. Occasional home health appliances are critical to keep going

Realistically it comes down to how conservative you are with over-provisioning. In the US we have the expectation of rarely to never tripping the main and when that happens it’s more likely an electrician call