Image Processing Project- Gesture driven Virtual Keyboard using OpenCV + Python

The Virtual Keyboard is something that I am very proud of.

The logic is really simple.

First you design the keyboard the way you want.

In my case, I made the keyboard so that it consisted of 26 alphabets and a space bar.

The designing is done by simple maths and nothing else. I mean first you decide the width and height of a key and where you are going to put the first key i.e 'q'.

According to the position of the first key you can keep on adding the key width to get the position of the next key.

And to get the position of the next row you just add the key height.

Now simply loop it.

You will get the keyboard.

The function that I used to make the keys is OpenCV's rectangle() function.

Now for the click gesture, the logic is simple here too.

First of all, what I do is Color Segmentation.

I wear a yellow (or any colored) paper in my finger.

Using color segmentation I can separate the yellow paper from the rest of the image.

Then I take the center of the paper. The center acts as a pointer for which key is to be pressed.

Now if you think carefully, if the area of the paper first increases and then decreases (for the camera), then it is a click.

The area increases if you bring the paper closer to the camera and decreases when you bring it away from it.

This is what my program detects. Of course there are some thresholds that are set or else every little change in area will be considered as a click.

The libraries needed for the Gesture driven Virtual Keyboard are:
• OpenCV
• Numpy
• Pyautogui.

There are some limitations though.

Firstly, fast typing is very hard.

Secondly, you need to wear a yellow paper on your finger which is sometimes uncomfortable.

P.S This image processing project was written by Dibakar Saha.

For more information about the project, a link would be provided on the comment section.