A method of testing a hearing ability of a user using a portable sound source playing apparatus includes outputting a signal indicating at least one phoneme, obtaining a response of a user regarding a degree of audibility of the output signal, and testing the hearing ability of a user based on hearing characteristics of the user showing the degree of audibility of a frequency band comprising a frequency of the at least one phoneme based on the obtained response.