Retard means literally “delayed”, ultimately from latin tardus (also see “tardy"), meaning slow. Doesn’t seem like it comes from the same suffix.