Defect control of magnetism in complex oxides