When I had a similar issue to this I resolved it by adjusting my retraction settings, specifically retraction distance and speed. I think that may be the case here, as the layers most dramatically affected appear to be those with the most retractions taking place. Depending on the material and the style of printer you have (direct drive or Bowden) your settings will be different, but I would start be dropping the retraction distance and see how that changes things.