In one aspect, a system for detecting accumulations of field materials between ground engaging components of an agricultural implement may include first and second ground engaging components configured to rotate relative to soil within a field as the implement is moved across the field. The first and second ground engaging components may be spaced apart from each other along an axial direction of the shaft. The system may also include a sensor configured to detect field materials within a detection zone defined directly between the first and second ground engaging components and above the axis of rotation in a vertical direction of the agricultural implement. Furthermore, the system may include a controller communicatively coupled to the sensor, with the controller configured to determine a parameter associated with an accumulation of the field materials between the first and second ground engaging components based on data received from the sensor.
