As someone who works with people in residential construction fairly often, this is the answer - and it’s why they don’t build new “starter” homes anymore. It’s very difficult to turn a profit on a single family home that would be considered affordable most places.

Basically, its very little extra effort and expense to build a luxury house compared to an inexpensive one, and your profit margin goes from very thin to decent.

Anecdotally in my area, most residential new construction is going to retirees who have a nest egg and the sale now very expensive house, or couples who sold an inherited house. Occasionally there are people who are remotely working or people building as an investment property, but they’re in the minority.