A magnetic impactor assembly is described herein. The magnetic impactor assembly generally includes a guide receptacle, and an impactor guide magnetically coupled to the guide receptacle to form a magnetic interface therebetween. The guide receptacle is attachable to a surgical device such as a surgical robotic manipulator arm. The impactor guide receives and guides an impactor to permit a user to impact a prosthesis into a bone of a patient in a planned position and orientation. The magnetic impactor assembly reduces the transmission of excessive forces to a patient or the surgical device if an off-axis impaction force is generated on the impactor through the decoupling of the impactor guide from the guide receptacle.