PROBLEM TO BE SOLVED: To provide an image processing device and a medical observation system capable of flexibly adjusting the brightness in a high brightness region in an image. SOLUTION: An imaging unit that enlarges an observation target and enables imaging, a first brightness adjustment mode that adjusts the brightness of a brightness adjustment mode with a preset sensitivity based on an observation state, and a sensitivity of the brightness adjustment. Is lower than the sensitivity of the first brightness adjustment mode, and the mode switching unit switches to one of the second brightness adjustment modes that suppress the brightness in the high brightness region, and the brightness adjustment according to the brightness adjustment mode switched by the mode switching unit. It is provided with a brightness adjusting unit for performing the above and an image processing unit for generating an image after the brightness adjustment by the brightness adjusting unit. [Selection diagram] Fig. 1【課題】画像中の高輝度領域に対して、柔軟に輝度調整を行うことができる画像処理装置及び医療用観察システムを提供すること。【解決手段】観察対象を拡大して撮像可能とする撮像部と、観察状態に基づいて、輝度調整モードを、予め設定された感度で輝度調整を行う第１輝度調整モード、及び輝度調整の感度が第１輝度調整モードの感度よりも低く、かつ高輝度領域の輝度を抑制する第２輝度調整モードのいずれかに切り替えるモード切替部と、モード切替部が切り替えた輝度調整モードに応じた輝度調整を行う輝度調整部と、輝度調整部による輝度調整後の画像を生成する画像処理部と、を備える。【選択図】図１
