Disclosed is a method of removing a background noise from a photoacoustic image, the method including applying an ultrasonic signal to a target absorbing body having a multi-modal microbubble contrast agent injected thereinto and receiving the ultrasonic signal reflected by the target absorbing body in order to acquire an ultrasonic image, when a photoacoustic signal is generated from the target absorbing body having the multi-modal microbubble contrast agent injected thereinto as the result of absorbing a laser pulse applied to the target absorbing body, receiving the photoacoustic signal in order to acquire a photoacoustic image, applying a critical value to pixels corresponding to microbubbles in the ultrasonic image in order to generate a mask image, and removing a background noise generated from a non-target absorbing body, while maintaining the target absorbing body, from the photoacoustic image using the mask image.