Brain tumor or other tissue classification and/or segmentation is provided based on from multi-parametric MRI. MRI spectroscopy, such as in combination with structural and/or diffusion MRI measurements, are used to classify. A machine-learned model or classifier distinguishes between the types of tissue in response to input of the multi-parametric MRI. To deal with limited training data for tumors, a patch-based system may be used. To better assist physicians in interpreting results, a confidence map may be generated using the machine-learned classifier.