Provided is an image processing apparatus, a method of processing an image, and an image processing program, which can clearly distinguish an abnormal portion protruding from a surface of a mucous membrane and a bubble, and can accurately detect the abnormal portion. An image processing apparatus 1 includes a contour extracting unit 110 that extracts a plurality of contour pixels from the image acquired by capturing an inside of a lumen of a living body, a feature data calculator 130 that calculates feature data based on pixel values of the plurality of contour pixels and positional relationship among the plurality of contour pixels, and an abnormal portion detector 140 that detects the abnormal portion based on the feature data.
