I believe there are settings related to minimum layer time. It will change cooling parameters or might slow the print down to achieve the minimum time.

That might make your smaller prints look bad.