The invention relates to a shelling and refining machine of camellia oleifera fruits. The shelling and refining machine comprises a refining belt conveying device arranged on a machine frame, whereinthe refining belt conveying device is connected with a shell-removing mechanism through a refining conveying trough; a refining trough body is arranged on the refining belt conveying device; a refining guide block and a refining spacing block are arranged in the refining trough body; a refining material sweeping block is arranged in the inner top part of the refining trough body close to the refining conveying trough; the space between the refining material sweeping block and the refining belt conveying device is bigger than one camellia oleifera seed and smaller than two camellia oleifera seeds; a refining port is formed in the refining spacing block; a refining push block is embedded into the refining trough body; the refining push block is connected with a refining air cylinder in a direction from the front side to the back side and arranged on the machine frame; a refining light beam transmitter is embedded into the inner side of the refining push block; and a reflected light beamreceiver is arranged on the inner side surface of the refining push block. Camellia oleifera seeds can be detected by the refining light beam transmitter and the reflected light beam receiver on the refining push block, thereby separating out inferior camellia oleifera seeds and realizing high-efficiency refining of the camellia oleifera seeds.