If you need to plug the headphones into the adapter, you can just leave them plugged in after disconnecting from the phone

This way, the headphones almost become ones with USB-C connectors than auxiliary barrels.