While I don't actually know a goddamn thing about the history of this, that doesn't seem to work too well once you look at more languages. While a male/female or male/female/neuter system is common in Indo-European languages, other language groups use versions that have more distinctions and haven't traditionally been associated with gender. Most languages in the Atlantic-Congo group that a lot of the southern half of Africa speaks have between ten and twenty different categories of noun in that sense. That's why they're more formally called "noun classes" rather than "grammatical genders"