The application relates to a listening device adapted for being located in or at an ear of a user and comprising a) an input transducer for converting an input sound to an electric input signal and b) an output transducer for converting a processed electric signal to an output sound a forward signal path being defined there between and comprising c) a signal processing unit for processing the electric input signal or a signal derived therefrom and providing a processed output signal d) a manually operable user interface located on the listening device allowing a user to control a function of the listening device e) a feedback estimation system for estimating a feedback path from the output transducer to the input transducer, the feedback estimation system comprising e1) an adaptive filter, the adaptive filter comprising e11) a variable filter part and e12) an algorithm part comprising an adaptive algorithm, the variable filter part being adapted for providing a transfer function to a filter input signal and providing a filtered output signal, the transfer function being controlled by filter coefficients determined in the algorithm part and transferred to the variable filter part, the feedback estimation system further comprising e2) an update control unit adapted for controlling the adaptive algorithm including the transfer of filter coefficients to the variable filter part. The application further relates to a method of operating a listening device, to its use and to a listening system. The object of the present application is to provide an improved control mechanism for an adaptive filter. The problem is solved in that the update control unit is adapted to monitor the manually operable user interface and to provide that an activation of the manually operable user interface is used for influencing the control of the adaptive algorithm. This has the advantage of providing a scheme for handling the risk of howl during manual operation of a listening device, e.g. a he
