One other thing that occurred to me is squish. Just like your first layer issues, the plastic on every layer needs something beneath it to squeeze against so it converts from a circular tube to a flat, wide oval. Maybe your slicing didn’t have enough support or infill in the main body as compare to the final top surface? You could try increasing the horizontal shells.

By the way, I assume you did not turn on ironing, because that could also affect the surfaces differently if one got ironed and the other didn’t for some reason.