International Journal of Scientific & Engineering Research, Volume 5, Issue 4, April-2014 650

ISSN 2229-5518

Virtual Keyboard

Nikhil Koul Pranav Nawathe Pranav Tulpule

Abstract- Our paper, the ‘virtual keyboard’ will discuss a new technology in human machine interface. Instead of using the mechanical keyboard having moving parts and buttons, this gadget projects an image of a keyboard for the user’s reference. Using a camera and digital image processing algorithms the user’s input is detected.

KEY TERMS- virtual keyboard, image processing, thresholding, projection, camera, touchscreen, MATLAB

1.INTRODUCTION

The modern keyboard for personal computers has been developed from a similar one used in typewriters. The layout has remained the same, but the computer keyboard uses the making and breaking of an electrical contact to detect the key-press. The major disadvantage of this concept is that a large amount of physical space is needed to accommodate the keyboard, making it unsuitable for applications such as mobile phones where it placed limitations on the screen size. To overcome this drawback, the touchscreens were developed, which integrated the input mechanism in the screen itself. However, typing on touchscreens is inconvenient for most users, due to the small size of buttons. Also, as the touchscreen typing keypads are integrated in the computer software, there are some security issues.
To solve both these problems we decided to build a virtual keyboard which would be able to project a full size keyboard from a small gadget combining a projector and a camera in a handheld package. The computers processor would be used to process the images obtained by the camera. As of now, we have been able to implement this concept using MATLAB software.
Our paper is organized as follows. The paper begins with introduction of keyboards and issues related to it. Next section elaborates the concept of Virtual Keyboard in which we have explained the operation of our concept. Then we describe the current status of implementation, followed by the future scope of this concept. The last part concludes our paper.

2.VIRTUAL KEYBOARD

Keyboards are input devices to computers comprising of keys that correspond to an input character or command. The term ‘Virtual’ means there is only a visual representation of the keyboard and it is not present physically. In our concept the visual representation is
actually a projection of the keyboard and user types in commands through that.
The implementation is done with the use of DIP (Digital Image Processing). So, there’ll be a device that will have a projector for projecting the keyboard and the camera which will be used for image acquisition purposes.

2.1THE BUILDING BLOCKS

The major components are:
1. Projector: used to project the keyboard image for the user’s reference.
2. Camera: used for capturing the image of the user’s
input.
3. Processor: used to process the image captured by
the camera. The computer’s processor is used.
4. Software: there will be a software written in .NET platform for Windows PCs and equivalent platforms for other computing devices. The purpose of this software will be frame extraction and processing it.
We are implementing it using MATLAB software for both frame extraction and image processing. The GUI is also implemented in MATLAB.

2.2 WORKING

There are four modules in the implementation of this concept:
1. Frame extraction module
2. Redundant data removal and information extraction module
3. Gridding module
4. Positioning module
All this is embedded with GUI.

2.2.1 The hardware setup will consist of a device (projector and camera) which will be connected to a computer (on

International Journal of Scientific & Engineering Research, Volume 5, Issue 4, April-2014 651

ISSN 2229-5518

which the keyboard is been implemented) through a USB
cable. The projector will be constantly projecting the
reference image of the keyboard.

2.2.2 This projection will also have 4 reference points at 4 corners. The user will also wear ‘tags’ on fingers.

2.2.3 Camera will be constantly taking the video of the projection and feeding to the software.

2.2.4 We generally keep our fingers above the keyboard and when we want to press any key we move the finger towards the key. The software detects this movement from the video feed and extracts the frame. The frame will contain all the four references and the tag on the finger. Position of the tag will be exactly on the key user want to press.

2.2.5 After the frame extraction, all the four references and tag pixel positions are extracted by thresholding first and then various positioning algorithms.

2.2.6 Now with the help of references gridding of the keyboard is done. In other words, keyboard that is used is created virtually in the code.

2.2.7 After gridding, the tag position is matched with every block in the grid and appropriate key is located.

2.2.8 This key is given back to the keyboard event running in the memory and it is displayed on monitor.

2.3 FLOW DIAGRAM

Frame Extraction

Binary Thresholding

Redundancy Removal

& Assignment

Gridding

Key Detection

GUI or Key Driver

Initialization

3. CURRENT STATUS OF IMPLEMENTATION

As we have mentioned earlier, we are trying to implement this concept practically. For the image processing algorithm development we have implemented it using MATLAB.

3.1 Implementation

3.1.1 In MATLAB, we have created a video object acquired by the webcam. This video is used for frame extraction.

3.1.2 Our MATLAB code constantly extracts 2 frames from video and with the help of Frame Differencing algorithm it monitors movement of the finger. If any movement is detected then next frame to that is extracted.

3.1.3 The extracted frame is firstly thresholded to get pixels of references and tag. Various data redundancy algorithms like addition of coordinates and elimination are run to extract 1 pixel per reference.

International Journal of Scientific & Engineering Research, Volume 5, Issue 4, April-2014 652

ISSN 2229-5518

Following is the figure of finger tag we are using

3.1.4 With the help of references, gridding is done. As of now we have successfully implemented keyboard with any combination up to 4X4. Our code has flexibility to allow a keyboard of any size with a mere change of 2 numbers.

3.1.5 After gridding, positioning of the tag is done. For positioning we use left upper and right lower points in the grid to detect corresponding key.

3.1.6 Next module is GUI. GUI is used to display the key which is detected. It is created using the GUIDE toolbox of MATLAB.

3.1.7 Following figure shows our setup:

3.2 Specifications

The primary aim of this concept was to produce a low-cost alternative to the commonly used keyboard. To maintain the low cost and due to unavailability of some components, the components and their specifications had to be compromised.
1. Projector: A suitable projector is unavailable at a low cost in the Indian market. To overcome this we used a keyboard drawn on paper. It has to be perfectly adjusted for all references in camera frame.
2. Camera: suitable camera available and is being
used for development. We used C170 webcam by
Logitech.
3. Processor: we are using processor of the computer.
We have used Sony Vaio F15218 laptop for testing and implementation.
4. Software: MATLAB R2010a has been installed on
the computer on which we tested the implementation.

4. FUTURE SCOPE

The basic idea of building a virtual keyboard can be expanded upon to build an even better and more user- friendly gadget.
In the short term the following changes can be made to the current concept:

4.1 Increase in Speed: The operating speed of the gadget can be increased by porting the entire image processing code to C or.NET.

4.2 Multi-touch compatibility: Our implementation supports only single touch. It can be extended to multi- touch by use of different colors for tags and color image processing. More sophisticated Image processing algorithms will be needed for better operation.

4.3 Android compatibility: The virtual keyboard can be adapted for Android based devices by creating a separate app as a device driver. However, more research needs to be done before this can be implemented as the processors of most mobiles or tablets are slower when compared to their PC counterparts. Image processing could use valuable resources on already resource starved device.

4.4 Integrating with DSP processor: A DSP processor can be integrated in the device itself to be used for image

processing. This would overcome the problem mentioned in the previous point, but would increase the cost of the gadget.

4.5 Building a complete touchscreen-like interface:

The entire image of the tablet or mobile screen or PC

International Journal of Scientific & Engineering Research, Volume 5, Issue 4, April-2014 653

ISSN 2229-5518

monitor could be projected on the surface. And as the concept uses techniques like gridding it could allow the user to not only have a virtual keyboard but also a
complete virtual touchscreen.

4.6 Elimination of finger tags: As of now, tags have to be worn on fingers. These tags could be eliminated through the use of sophisticated DIP algorithms, such as feature extraction.

5. CONCLUSION

The virtual keyboard concept demonstrated in our project is not only technically feasible, but also has enormous market potential. As our aim for building the gadget was only to test its technical feasibility, there are some drawbacks to the device which need to be overcome before its true potential can be realized.

6. ACKOWLEDGEMENT

We wish to thank our project guide Prof. Mrs. K.J.Kulkarni
, Professor, E & Tc Department, PVG’s COET, Pune for her invaluable guidance and support.

7. REFERENCES

[1] Huan Du Thierry Oggier, Felix Lustenberg, “Virtual

Keyboard Based on 3D optical ranging”,

[2] Prof. A.S.Mane, Prof. S.M.Sangve ,”Reconfigurable Virtual

Keyboard”

[3] Rafael C. Gonzalez and Richard E. Woods , “Digital Image

Processing”

[4] Rafael C. Gonzalez and Richard E. Woods, “Digital Image

Processing using MATLAB”

[5] Sergey Simakov, “Introduction to MATLAB graphical user interfce”

[6] www.mathworks.in/matlabcentral

[7] www.peripherals.about.com

Author Details:

Nikhil Koul- nikhilkoul12@gmail.com
Pranav Nawathe – pranav.nawathe92@gmail.com
Pranav Tulpule – pranavtulpule@gmail.com