Comment on The Economist on using phrenology for hiring and lending decisions: "Some might argue that face-based analysis is more meritocratic" […] "For people without access to credit, that could be a blessing"

panda_abyss@lemmy.ca ⁨4⁩ ⁨days⁩ ago

• GenderNeutralBro@lemmy.sdf.org ⁨3⁩ ⁨days⁩ ago

  The problem here is education.

  And I’m not just talking about “average joes” who don’t know the first thing about statistics. It is mind-boggling how many people with advanced degrees do not understand the difference between correlation and causation, and will argue until they’re blue in the face that it doesn’t affect results.

  AI is not helping. Modern machine learning is basically a correlation engine with no concept of causation. The idea of using it to predict the future is dead on arrival. The idea of using it in any prescriptive role in social sciences is grotesque; it will never be more than a violation of human dignity.

  Billions upon billions of dollars are being invested in putting lipstick on that pig. At this point it is more lipstick than pig.

  source
• Jason2357@lemmy.ca ⁨2⁩ ⁨days⁩ ago

  I cant imagine a model being trained like this /not/ end up encoding a bunch of features that correlate with race. It will find the white people, then reward its self as the group does statistically better.

  source

  • CheeseNoodle@lemmy.world ⁨2⁩ ⁨days⁩ ago

    Even a genuinely perfect model would immediately skew to bias; the moment some statistical fluke gets incorporated into the training data that becomes self re-enforcing and it’ll create and then re-enforce that bias in a feedback loop.

    source

    • Jason2357@lemmy.ca ⁨2⁩ ⁨days⁩ ago

      Usually these models are trained on past data, and then applied going forward. So whatever bias was in the past data will be used as a predictive variable. There are plenty of facial feature characteristics that correlate with race, and when the model picks those because the past data is racially biased (because of over-policing, lack of opportunity, poverty, etc), they will be in the model. Guaranteed. These models absolutely do not care that correlation != causation. They are correlation machines.

      source
• ssladam@lemmy.world ⁨2⁩ ⁨days⁩ ago

  Exactly. It’s like saying that since every president has been over 6’ tall we should only allow tall people to run for president.

  source