A radiation therapy system (1) includes an ultrasound (US) imaging unit (2), a registration unit (30), an US motion unit (44), and a real-time dose computation engine (46). The ultrasound (US) imaging unit (2) generates a baseline and real-time US images (3) of a subject body (4) region including a target and one or more Organs At Risk (OARs). The registration unit (30) deformably registers a planning image (32) and the baseline US image (36), and maps (66) radiation absorptive properties of tissue in the planning image (32) to the baseline US image (36). The US motion unit (44) measures motion of the target volume and OARs during radiation therapy treatment based on the real-time US images. The real-time dose computation engine (46) computes a real-time time radiation dose delivered to the tissues based on the tissue radiation absorptive properties mapped from the baseline or planning images to the real-time 3D US images (3).