A method and system for assisting a physician track a surgical device in a body organ of a subject during a procedure includes fluoroscopic based registration, tracking, and optimizing a fluoroscopy position. An initial registration step includes receiving a 3D image data of a subject in a first body position, receiving a real time fluoroscopy image data, and estimating a deformation model or field to match points in the real time fluoro image with a corresponding point in the 3D model. A tracking step includes computing the 3D location of the surgical device and displaying the surgical device and the 3D model of the body organ in a fused arrangement. Optimizing the fluoroscope camera pose includes computing a candidate camera pose to assist the surgeon to track a surgical device based on features of the surgical device, position of the patient, and mechanical properties or constraints of the fluoroscope.
