Disclosed herein is a computer-aided diagnosis system and a computer-aided diagnosis method. This method includes the step of performing a principal component analysis to acquire an effect of light and shade portions from a color image of skin to serves as a first principal component. This method performs the principal component analysis further to acquire a second principal component and a third principal component, and the second and third principal components contain color variability. The third principal component is correlated with a skin cancer and serves as a main indicator of variegated colors for malignancy diagnosis.