Not necessarily? Icy water would have been rare, but even in the summer water from a stream or a lake is colder than the air, which is enough to make it refreshing. And well water can be very cold.