This used to be my job. They're not controlling the cars. They're basically completing real-time CAPTCHAs, telling the car whether the cameras see a stop sign, a bicycle, temporary barriers, etc. If the car can't identify an object that could possibly cross its path, it pulls over and stops until an operator can do a sanity-check on whatever the car's confused by. They only need to be able to identify objects on the road, not know the rules of the road.