This is kind of the thing.

Having DEI committees is now one of several symbols of the political ideologies of a company and its “senior leadership”.

For both sides, I think a lot of people (or at least those that have the privilege of choosing where they work) do not want to work for a company that directly conflicts with their political leanings.

A far left worker who sees their employer axing their DEI programs could see that as a symbol that the company is swinging hard to the right, and may adapt more conservative practices that may effect them directly. See: Hobby Lobby trying to block their health insurance from covering birth control.

Likewise, a far right worker who sees their employer adopting a leftist program like DEI might start to get concerned that their employer may also start swinging more to the left and adopt progressive practices that might impact them directly. Like paying them a living wage, or hiring a queer person.