Rookie mistake, the thing holding the magnet from the back side is also metal so it cancels out