Not being able to visualise images has no bearing on whether you can conceptualise surrounding space (positions, distances) or you wouldn’t be able to function at all. While aphantasia is not a medically recognised condition and therefore not exactly well studied there’s a lot to suggests it partially overlaps with autism. This in turn would explain tendency for bottom-up thinking - your brain has to systemise everything to understand it. At least that’s my intuition as someone with both.