It’s up to each app developer whether they wanna use the new API. According to the docs linked in this thread, each app needs to ask the user separately which option they prefer (so as of right now, there isn’t a system wide option for it).

So it’s not very clear to me why that option exists, even as a developer setting. Maybe Android itself uses it for its UI, but it’s still in the testing phase? Someone who has their phone set to one of those gendered languages could test it to see if changing the setting actually changes anything.