system which allows free mobility of the user. This tracker is interfaced with a laptop/desktop system running image
International Journal of Scientific & Engineering Research, Volume 3, Issue 10, October-2012 1
ISSN 2229-5518
Eye Controlled Wheelchair
Sandesh Pai, Sagar Ayare, Romil Kapadia
Abstract—This paper delivers a new method to guide and control the wheelchair for disabled people based on ‘Human-Computer Interaction (HCI). An eye control device based on image processing is developed to assist mobility. The eye movement is detected using a camera which is attached to the wheelchair. This concept can be used for multiple applications, but this paper focuses the application to mobile and communication aid for handicapped patients suffering from quadriplegic and paraplegic diseases. The proposed system involves two stages; first eye movement detection and second, sending of control signals to the powered wheelchair.
Index Terms—Muscular Dystrophy, Pupil, fundus, electro-oculogram, microlens camera
—————————— ——————————
wheelchair is mobility-aided device for persons with moderate/severe physical disabilities or chronic diseases as well as the elderly. In order to take care for different disabilities, various kinds of interface have been developed for wheelchair control: such as joystick control, head control. However, to steer their own wheelchair through a conventional joystick is difficult for people who experience total paralysis in all four limbs, such as muscular dystrophy (MD), spinal cord injury (SCI), amyotrophic lateral sclerosis
(ALS), etc.
Clinicians indicated that 9 to 10 percent of patients who receive wheelchair training find it extremely difficult or impossible to use the wheelchair for activities of daily living. Table 1 shows the results of a survey taken of disabled people who face problem while doing the Activities of Daily Life (ADL). The results clearly suggests that approximately 40 % of the disabled patients, even after having training with powered wheelchair, have difficulty with steering tasks such as maneuvering their wheel chair
The results indicate a need, not for innovative steering interfaces, but for entirely new technology for autonomous navigation, eye controlled is one of such technology.
————————————————
Sandesh Pai has pursued bachelor's degree in Electronics & Telecommunications engineering from University of Mumbai ,India, PH-
+91-8861629896. E-mail: paisandesh@ymail.com
Sagar Ayare has pursued bachelor's degree in Electronics &
Telecommunications engineering from University of Mumbai ,India, PH-
+1-832-4557722. E-mail: sagarayare87@gmail.com
Romil Kapadia has pursued bachelor's degree in Electronics &
Telecommunications engineering from University of Mumbai ,India, PH-
+91-98193 80026. E-mail: kapadiaromil@yahoo.co.in
TABLE 1
STATISTICAL ANALYSIS BY DEPARTMENT OF VETERANS AFFAIRS
Measure | Simple Average | Weighted* Average |
% of patients who, after training, have extremely difficulty using a power wheelchair | 6 | 6* |
% of patients who, after training, find it impossible to use a power wheelchair | 4 | 3* |
Total Percentage | 10 | 9 |
Percent of power wheelchair users who have difficulty with steering tasks | 32 | 55** |
Percent of power wheelchair users for whom steering tasks are impossible without assistance | 9 | 5** |
Total Percentage | 41 | 40 |
*weighed by number trained; **weighed by total number of power wheelchair users |
There are there eye tracking mechanisms to compute the position of pupil.
The Electro-Oculogram method obtains the gaze direction by sensing the electrooculographic potential. This is done by measuring the potential using electrodes placed on face where human eye is an electric dipole with a negative pole at the fundus and positive pole at the cornea.
IJSER © 2012
International Journal of Scientific & Engineering Research, Volume 3, Issue 10, October-2012 2
ISSN 2229-5518
system which allows free mobility of the user. This tracker is interfaced with a laptop/desktop system running image
C processing software to resolve the eye movement.
A B
D
E
Fig. 1. Electro-Oculogram Method
As shown in the Fig. 1, one electrode is placed to the side of the left eye and another to the side of the right eye. This pair shall detect horizontal eye movements. One electrode is placed above the left eye and another below the left eye. This pair shall detect vertical eye movements. A fifth electrode is attached by the ear to provide reference voltage.
These electrodes send the electrical signals to two EOG
circuits of similar design to detect the horizontal and vertical
movement of the pupil. This information is sent for
computation. The big advantage of this method is the ability to
detect eye movements even when they are closed.
In this method a non slipping contact lens fits over corneal bulge. The tracking of the pupil is recorded by affixing a magnetic coil or mirror to the lens.
Fig. 2. Lens tracking System
The integrated mirror in the contact lens allows measuring reflected light; alternatively, the integrated coil in the contact lens allows detecting the coil’s orientation in magnetic field. The big advantage of this method is high accuracy and nearly unlimited resolution in time.
Both methods explained so far are obtrusive and are not suited well for interaction by gaze. The third and preferred method for eye-gaze interaction is using video camera.
The most common mechanical setup involves use of desktop computer with integrated eye tracker camera. The desktop computer is loaded with software package for analyzing the eye gaze data. This setup require head fixation and hence restricts the head movements.
An alternative method uses a head mounted eye tracking
Fig. 3. Lens tracking System
The block diagram shown in Fig. 4 shows the block diagram of the Eye Controlled Wheelchair. The eye movement is tracked using a camera mounted on to the headgear.
Fig. 4. Block Diagram
The headgear construction greatly simplifies the optics by using a microlens video camera for dark-pupil tracking. The headgear is equipped with proper illumination. This headgear is interfaced to the laptop/desktop via USB interface through which real time video of the eye is sent. The laptop/desktop hosts image processing software which processes the video feed from the camera and determines the position of the pupil. The laptop/desktop then sends signals corresponding to the pupil position to the micro-controller circuitry which drives the motor through motor driving circuit.
Opto-Isolator is an electronic device designed to transfer
IJSER © 2012
International Journal of Scientific & Engineering Research, Volume 3, Issue 10, October-2012 3
ISSN 2229-5518
electric signal by utilizing optical light waves to provide coupling with electric isolation between the input and output. The main purpose of the opto-isolator is to prevent high voltage or rapidly changing current from damaging the serial port of the laptop/desktop.
Fig. 5. Opto-Isolator
The signals coming from the opto-isolator is then connected to the micro-controller. Based on the signals received by the micro-controller, it sends the control signal to the wheelchair controller.
Wheelchair Controller consists of two motor driving circuits, one each for the rear wheels. The motor driving circuit gets the input from the micro-controller.
Fig 6. Motor Driving Circuit
Fig. 6 shows driving circuit for one of the motors. The inputs A1 and A2 are connected to the micro-controller output. The status of signal at A1 and A2 determines the polarity across the motor ‘M’, which determines the direction of motor rotation viz. clockwise or anti-clockwise. The connection of the motor is such that ‘1 0’ status for A1 A2 moves the wheel in clockwise direction and ‘0 1’ status for A1 A2 moves the motor in anti-clockwise direction.
The software used to determine the position of the pupil is Matlab which is installed in the laptop/desktop system. The real time feed provided by the camera is processed by the algorithm depicted below.
1. The video from the camera is divided into frames.
2. Each frame is divided into 3x3 sectors and the frame image is converted to gray scale.
3. The intensity level is adjusted so that only the black pupil is visible.
4. The sector in which the pupil is present is recorded
5. A counter is set to count the number of consecutive frames the pupil remains in the sector
6. If the counter reaches a preset value, the a unique
signal is sent to the micro-controller through the serial port.
Fig 7. Output of the Image Processing Software at various stages
The signal received from the serial port is the processed by the micro-controller and corresponding control signals are sent to the wheelchair controller. This program embedded into the micro-controller will move the motor in either clockwise or anti-clockwise direction.
IJSER © 2012
International Journal of Scientific & Engineering Research, Volume 3, Issue 10, October-2012 4
ISSN 2229-5518
5. RESULT
The micro-controller is interfaced with the wheelchair controller as shown in Fig. 8
Fig 8. Micro controller interfacing with motor driver circuit
The micro-controller sends of the signals to the wheelchair controller so as to get following results.
The wheelchair moves in the forward direction if the left wheel moves in the clockwise direction and the right wheel moves in the anti-clockwise direction. Thus the status of the output port A1 A2 A3 A4 is ‘1 0 0 1’.
The wheelchair moves in the forward direction if the both the wheels move in the anti-clockwise direction. Thus the status of the output port A1 A2 A3 A4 is ‘0 1 0 1’.
Fig 11. Left Motion
This system consists of a camera, micro-controller and wheelchair controller. This hardware along with the software can prove to be an effective system to make the life of the paraplegic patients independent.
The critical part of the system is image processing at real time which can be addressed by using better high end image processing software
The system can be extended to control the equipments around such fans, lights, etc. The software can also be modified, and along with sound synthesis can be used to generate voice commands.
Fig 9. Forward Motion
[1] Journal of Rehabilitation Research and Development Vol . 37 No . 3, May/June 2000 Pages 353—360 by Department of Veterans Affairs
[2] Wooding, D.S. (2002). Fixation Maps: quantifying eye-movement traces. In: Proceedings of the Eye Tracking Research & Applications Symposium 2002. 31-36.New York, ACM.
[3] Salvucci, D.D. & Goldberg, J.H. (2000). Identifying fixations and
saccades in eye-tracking protocols. In: Proceedings of the Eye Tracking
The wheelchair moves in the forward direction if the both the
wheels move in the clockwise direction. Thus the status of the
output port A1 A2 A3 A4 is ‘1 0 1 0’.
Fig 10. Right Motion
Research and Applications Symposium 2000,71-78. NY: ACM Press.
[4] “OpenEyes: a low-cost head-mounted eye-tracking solution” [Li, Babcock,
Parkhurst 2006]
[5] Raj kamal,Microcontrollers: Architecture, Programming, Interfacing and System Design, Prentice Hall.
[6] www.mathwork.com
IJSER © 2012
International Journal of Scientific & Engineering Research, Volume 3, Issue 10, October-2012 5
ISSN 2229-5518
IJSER 2012
http:llv¥ww.ijserorq