The invention relates to a microscope or endoscope assembly (1), in particular for a surgical microscope such as an ophthalmic microscope. Especially in eye surgery, but also in other application concerning life tissue, specular reflections of the illumination light are unwanted because they may hide important information contained in a diffuse reflection at the same location. In order to suppress such unwanted specular reflection, the microscope or endoscope assembly according to the invention comprises an observation region (4), in which an object (6) to be observed, such as an eye, can be arranged. The assembly further comprises an illumination light path (8) which extends from an illumination light entry region (24), where illumination light enters into the assembly, to the observation region. Further, an observation light path (10) is comprised, which extends from the observation region (4) to a light exit region (34), where the light leaves the assembly and may be collected by an observation subassembly (36), such as at least one camera and/or at least one ocular. In the illumination light path, a first polarizing subassembly (16) is arranged, to create polarized illumination light (18). In the observation light path (10) a second polarizing subassembly (32) is arranged, which is configured to filter out the polarized illumination light (18) passing the first polarizing subassembly (16). To ensure coaxial illumination and observation of the object, a beam splitter (30, 40) is provided, which is arranged in the illumination light path (8) between the light entry region (24) and the observation region (4) and in the observation light path (10) between the observation region (4) and the light exit region (34). By the complementary filtering action of the first and second polarizing subassembly, specular reflections by the object can be suppressed and/or eliminated.