Devices and methods are provided for controlled application of a treatment to tissue adjacent a body cavity, such as after removal of tissue, e.g. cancer. A device embodying features of the invention includes one or more radiation shielding components to control emitted radiation from a radiation source to minimize radiation damage to healthy portions of the body cavity. A device embodying features of the invention can include a sealing member at a location on a shaft of the device proximal to a treatment location therein to seal the passageway leading to the body cavity. Methods for treating a body cavity include methods for delivering a radiation source to a body cavity while minimizing damaging irradiation of healthy tissue.