An ultrasound system with a deep learning neural net feature is used to automatically identify image anatomy or pathology and the view of the anatomy seen in the image. The feature also can assess image quality in real time. Based on identified anatomy, the system can automatically annotate images, launch measurement tools and exam protocols, and perform image control adjustments to aid diagnosis and improve exam workflow.