Absolutely. The vast majority of my sites do just fine when whitelisting only the primary domain. I consider it an essential add-on myself.

Lemmy is one of the few that needs a little babysitting, and it’s only for the purpose OP stated.