An apparatus and method for performing image segmentation. The method includes the steps of filtering an X-ray image of a jaw of a patient to reduce noise. Each image pixel is clustered into one of a lesion region and a background region. Further, the method transforms each pixel included in the clustered image into a neutrosophic domain determined by computing a three-tuple including a first probability corresponding to the pixel being correctly clustered in the lesion region, a second probability corresponding to the pixel being incorrectly clustered, and a third probability corresponding to the pixel being indeterminate, and iteratively computes the first probability, the second probability, and the third probability based on a mean value of pixel intensity of a predetermined window and a homogeneity value. The image is segmented by assigning the indeterminate pixels the lesion region or the background region based on a degree of pixel indeterminacy.
