An authentication system controlled by eye open and eye closed state and a handheld control apparatus thereof are provided. The handheld control apparatus includes a shell, an image capturing unit and a processing unit. The shell has a window and is suitable for a user to hold. The image capturing unit is disposed in the shell and captures an eye area of the user through the window to obtain an image sequence. The processing unit is coupled to the image capturing unit and analyzes the image sequence to obtain eye image information of the eye area of the user. The processing unit detects an eye-open state and an eye-closed state of the user based on the eye image information, converts a plurality of the eye-open states and the eye-closed states into an blink code, and accordingly generates a control command to control a security equipment.
