Conventional systems and methods of classifying heart signals include segmenting them which can fail due to the presence of noise, artifacts and other sounds including third heart sound ‘;S3’;, fourth heart sound ‘;S4’;, and murmur. Heart sounds are inherently prone to interfering noise (ambient, speech, etc.) and motion artifact, which can overlap time location and frequency spectra of murmur in heart sound. Embodiments of the present disclosure provide parallel implementation of Deep Neural Networks (DNN) for classifying heart sound signals (HSS) wherein spatial (presence of different frequencies component) filters from Spectrogram feature(s) of the HSS are learnt by a first DNN while time-varying component of the signals from MFCC features of the HSS are learnt by a second DNN for classifying the heart sound signal as one of normal sound signal or murmur sound signal.