I’m reminded of an early model that was trained to find if tanks were hiding pictures of forests / jungles. Was doing great with the training data then was given new images and seemed to be guessing wildly.

Turns out it in the training data all the pictures with tanks were taken on cloudy days.