A system and method combines information from a plurality of medical imaging modalities, such as PET, CT, MRI, MRSI, Ultrasound, Echo Cardiograms, Photoacoustic Imaging and Elastography for a medical image guided procedure, such that a pre-procedure image using one of these imaging modalities, is fused with an intra-procedure imaging modality used for real time image guidance for a medical procedure for any soft tissue organ or gland such as prostate, skin, heart, lung, kidney, liver, bladder, ovaries, and thyroid, wherein the soft tissue deformation and changes between the two imaging instances are modeled and accounted for automatically.
