16-09-2016, 11:30 AM
[attachment=71086]
Abstract: Input to small devices is becoming an increasingly crucial factor in development for the ever-more powerful
embedded market. Thereforethis paper presents an innovative approach towards virtual keyboard for consumers of
mobile, laptops, personal computer, tablet or any industrial machine. A virtual keyboard can be defined as a touch
typing device that does not have a physical manifestation of the sensing areas.This paperuses an inexpensive technique
to transform an ordinary piece of paper into a touch screen using an ordinary camera. Here we are using different image
processing algorithms to accurately recognize finger touch such as segmentation, thresholding etc.The keyboard layout
is reconfigurable,hence allowing user to change the layout based onapplication, for example user can select different
languagefor editor or select a specialized layout for gamingapplication.
INTRODUCTION
Touch typing or machine writing was invented
formechanical typewriters which had the current
QWERTY keylayout since 1874.This QWERTY interface
survived becauseof its many positive aspects. QWERTY is
the most common keyboard layout on English-language
computer and typewriter keyboards. It takes its name from
the first six characters present in the far left of the
keyboard's top first row of letters.
However, this is not feasible fortext entry in smaller
computing devices such as PDA’s andmobile phones and
input to the small devices is becoming anincreasingly
crucial factor in development for ever-morepowerful
embedded market [1].Touch typing is an input method that
employs discrete sensors, or sensed area, or buttons for
one or a set of atomic symbols (letters, digits, characters)
of a language. Examples are the common keyboard, the
keypad of a mobile phone and on-screen keyboards on
PDAs. This definition explicitly includes virtual buttons
that only differ from the surrounding physique in that their
extent is sensed by some technique for touch by a finger or
pointer [1].
A virtual keyboard can be defined as a touch typing device
that does not have a physical appearance of the sensing
areas i.e. the sensing area which acts as a button is not per
se a button but is programmed to act as one. Therefore,
sensing area can be realized with photo-electric sensors,
active finger tracing methods, or a touch pad. The latter is
different from a keypad as it does not have aprioridesignated
areas for buttons. Virtual keyboards that
employ discrete sensing area for each symbol re(rather
than cording methods) inherently allow the realization of
soft keyboard [1].
Up-till now, many designs of virtual keyboard have
beenproposed and implemented: finger-joint gesture
wearable keypad [2], visual panel [3], thumb code
chording glove [5], fingering [6], touch stream, multi point
touch pad, V Type, combined projection and recognition
virtual keyboard [7], sense board etc.
II. VIRTUAL KEYBOARD
As the demand for computing environments evolves, new
human-computer interfaces have been implemented to
provide multiform interactions between users and
machines. Nevertheless, the basis for most humantocomputer
interactions remains the binomial keyboard or
mouse. We are presenting here a technology for the next
generation, which is the Virtual Keyboard. As the name
suggests the virtual keyboardhas no physical appearance.
Virtual keyboard is an application that virtualizesthe
hardware keyboard with different layouts.
Recent advancements in the Information and
Communication Technologies (ICT) open up a scope for
computing in different languages. Off late, computing
devices are evolved in different shapes like cell phone,
PDA, iPod etc. In these devices, it is not possible to afford
hardware keyboard because of the problems in size and
weight of devices. To alleviate this problem, the
application developers propose virtual keyboard.
Virtual keyboard enables the user to type on any surface,
including a plain paper on your desk. Some virtual
keyboards give vibration as the in feedback; some are
projected on the typing surface, while others give different
kind of visual feedback such as showing it on a smart
phone’s screen. The user “presses” the virtual keys thus
typing the desired input text. The proposed solution is
applicable to any surface. The system has been
implemented on a computer, operates in real time, and
gives excellent results.
Enormous advancement in communication and
information technology instantiates the flourishment of the mobile and handheld devices in urban and rural areas in
India. These devices become popular to the common
people in India.In these devices, text entry task is not
possible throughconventional hardware keyboard due to
mobility restriction,size factor etc. This motivates
researchers to propose avirtual keyboard model which
imitates the hardware keyboardambiance for an effective
text entry system. Further, theutilization of virtual
keyboard appears in space savingsituations and also can be
extended into requirement in virtualprogrammability of
keys or systems avoiding mechanicalfailure or in
movement situations where usability of standardkeyboard
is limited. In these days, virtual keyboards findtheir
position in transport environments e.g. rail, plane
orautomotive. Virtual keyboards are also designed for
publickiosks. This reveals the fact that virtual keyboard
has becomea very essential needs in every area like
performing text entry,simulating hardware keyboard, and
adequately in accessingInternet to perform different kind
of tasks like E-mailing,chatting, blogging etc
This paper describes an improved method implementing a
virtual keyboard using the single integrated standard two
dimensional (2D) cameras. Camera based virtual
keyboards can be implemented using a single or multiple
cameras. One of the major challenges is how to determine
ifthe finger touches the surface or not. Touch detection
based on real three-dimensional (3D)model built from
stereoscopic camera based systemsis more accurate than
single camera based solutions. However, since
stereoscopic cameras are not common in mobile phones,
therefore this method isless applicable to mobile
solutions.Also the challenge of accurate touchdetection is
even greater when using a single camera and almost any
surface.Floating virtual keyboards, which are portableand
enabledirectingthe camera to any surface, are even more
challengingto implement.The proposed solution is based
on single standard camera; and presents an improved
touch detection method, based on microphone input. Thus
enables working on any surface or paper, as long as both
the camera input and microphone input are there.
The set-up is like this- first the camera is positioned
accordance with the paper which acts as the keyboard and
the microphone which is attached to the paper is interfaced
with the pc with the help of microcontroller. The user has
to apply red nail paint on the finger nails. Now as the user
starts typing in front of the camera and touches the paper,
the sound generated by tap of finger on keyboard is
detected by controller and 1/0 is sent to PC by serial
communication. The Matlab code triggers the camera and
the photo is captured. Then by applying different image
processing operations like segmentation, thresholding etc.
the required character is displayed on the text editor.
Basically image processing is any form of signal
processing for which the input is any form of an image,
such as a photograph or video frame; the output of an
image processing may be either an image or, a set of
characteristics or parameters related to the image. The
Image processing usually refers to digital image
processing, but optical and analog image processing can
also be done. With the help of this proposed system, the
keyboard layout can be generated using a paper. The
image being displayed on the paper is scanned via a
camera connected to the computer.
An image processing application will then detect the key
press events by user and process them accordingly. The
application must also generate virtual key press events for
OS as if a normal keyboard is used. The keyboard layout
can be changed hence allowing user to change the layout
based on application, for example user can select different
languages for editor or a specialized layout for gaming
applications. User can even design his own layout in
hardware version. User must also be able to reconfigure
the keyboard. i.e. the application must allow the user to
map keys based on his preference. This would allow the
user to set the keypad layout according to his
requirements.
There are 3 modules:
Image Capturing
Processing
Display
These modules do the processing when the layout is
provided the module does some processing on the image
captured by the camera. After processing this layout is
stored temporarily. When the user select character from
the keyboard and gives some input, the character is
captured by the camera, processing is done on it, location
of the character is checked. The provided input and the
stored character is compared and the desired output is
displayed on the screen.
Image Segmentation- It is defined as the process of
partitioning a digital image into multiple segments (sets of
pixels, which known as superpixels). The goal of
segmentation is to simplify and/or change the
representation of an image into something that is more
meaningful and easier to analyze. Image segmentation is
typically used to locate objects and boundaries (lines, curves, etc.) in images. More precisely, image
segmentation is the process of assigning a label to every
pixel in an image such that pixels with the same label
share certain characteristics.
The result of image segmentation is a set of segments that
collectively cover the entire image, or a set of contours
extracted from the image. Each of the pixels in a region
are similar with respect to some characteristic or computed
property, such as color, intensity, or texture. Adjacent
regions are significantly different with respect to the same
characteristic(s). When applied to a stack of images,
typical in medical imaging, the resulting contours after
image segmentation can be used to create 3D
reconstructions with the help of interpolation algorithms
like marching cubes.
Thresholding-The simplest method of image segmentation
is called the thresholding method. This method is based on
a clip-level (or a threshold value) to turn a gray-scale
image into a binary image. There is also a balanced
histogram thresholding.The key of this method is to select
the threshold value (or values when multiple-levels are
selected). Several popular methods are used in industry
including the maximum entropy method, Otsu's method
(maximum variance).Recently, methods which have been
developed for thresholding computed tomography (CT)
images. The key point is that, unlike Otsu's method, the
thresholds are derived from the radiographs instead of the
(reconstructed) image.
Abstract: Input to small devices is becoming an increasingly crucial factor in development for the ever-more powerful
embedded market. Thereforethis paper presents an innovative approach towards virtual keyboard for consumers of
mobile, laptops, personal computer, tablet or any industrial machine. A virtual keyboard can be defined as a touch
typing device that does not have a physical manifestation of the sensing areas.This paperuses an inexpensive technique
to transform an ordinary piece of paper into a touch screen using an ordinary camera. Here we are using different image
processing algorithms to accurately recognize finger touch such as segmentation, thresholding etc.The keyboard layout
is reconfigurable,hence allowing user to change the layout based onapplication, for example user can select different
languagefor editor or select a specialized layout for gamingapplication.
INTRODUCTION
Touch typing or machine writing was invented
formechanical typewriters which had the current
QWERTY keylayout since 1874.This QWERTY interface
survived becauseof its many positive aspects. QWERTY is
the most common keyboard layout on English-language
computer and typewriter keyboards. It takes its name from
the first six characters present in the far left of the
keyboard's top first row of letters.
However, this is not feasible fortext entry in smaller
computing devices such as PDA’s andmobile phones and
input to the small devices is becoming anincreasingly
crucial factor in development for ever-morepowerful
embedded market [1].Touch typing is an input method that
employs discrete sensors, or sensed area, or buttons for
one or a set of atomic symbols (letters, digits, characters)
of a language. Examples are the common keyboard, the
keypad of a mobile phone and on-screen keyboards on
PDAs. This definition explicitly includes virtual buttons
that only differ from the surrounding physique in that their
extent is sensed by some technique for touch by a finger or
pointer [1].
A virtual keyboard can be defined as a touch typing device
that does not have a physical appearance of the sensing
areas i.e. the sensing area which acts as a button is not per
se a button but is programmed to act as one. Therefore,
sensing area can be realized with photo-electric sensors,
active finger tracing methods, or a touch pad. The latter is
different from a keypad as it does not have aprioridesignated
areas for buttons. Virtual keyboards that
employ discrete sensing area for each symbol re(rather
than cording methods) inherently allow the realization of
soft keyboard [1].
Up-till now, many designs of virtual keyboard have
beenproposed and implemented: finger-joint gesture
wearable keypad [2], visual panel [3], thumb code
chording glove [5], fingering [6], touch stream, multi point
touch pad, V Type, combined projection and recognition
virtual keyboard [7], sense board etc.
II. VIRTUAL KEYBOARD
As the demand for computing environments evolves, new
human-computer interfaces have been implemented to
provide multiform interactions between users and
machines. Nevertheless, the basis for most humantocomputer
interactions remains the binomial keyboard or
mouse. We are presenting here a technology for the next
generation, which is the Virtual Keyboard. As the name
suggests the virtual keyboardhas no physical appearance.
Virtual keyboard is an application that virtualizesthe
hardware keyboard with different layouts.
Recent advancements in the Information and
Communication Technologies (ICT) open up a scope for
computing in different languages. Off late, computing
devices are evolved in different shapes like cell phone,
PDA, iPod etc. In these devices, it is not possible to afford
hardware keyboard because of the problems in size and
weight of devices. To alleviate this problem, the
application developers propose virtual keyboard.
Virtual keyboard enables the user to type on any surface,
including a plain paper on your desk. Some virtual
keyboards give vibration as the in feedback; some are
projected on the typing surface, while others give different
kind of visual feedback such as showing it on a smart
phone’s screen. The user “presses” the virtual keys thus
typing the desired input text. The proposed solution is
applicable to any surface. The system has been
implemented on a computer, operates in real time, and
gives excellent results.
Enormous advancement in communication and
information technology instantiates the flourishment of the mobile and handheld devices in urban and rural areas in
India. These devices become popular to the common
people in India.In these devices, text entry task is not
possible throughconventional hardware keyboard due to
mobility restriction,size factor etc. This motivates
researchers to propose avirtual keyboard model which
imitates the hardware keyboardambiance for an effective
text entry system. Further, theutilization of virtual
keyboard appears in space savingsituations and also can be
extended into requirement in virtualprogrammability of
keys or systems avoiding mechanicalfailure or in
movement situations where usability of standardkeyboard
is limited. In these days, virtual keyboards findtheir
position in transport environments e.g. rail, plane
orautomotive. Virtual keyboards are also designed for
publickiosks. This reveals the fact that virtual keyboard
has becomea very essential needs in every area like
performing text entry,simulating hardware keyboard, and
adequately in accessingInternet to perform different kind
of tasks like E-mailing,chatting, blogging etc
This paper describes an improved method implementing a
virtual keyboard using the single integrated standard two
dimensional (2D) cameras. Camera based virtual
keyboards can be implemented using a single or multiple
cameras. One of the major challenges is how to determine
ifthe finger touches the surface or not. Touch detection
based on real three-dimensional (3D)model built from
stereoscopic camera based systemsis more accurate than
single camera based solutions. However, since
stereoscopic cameras are not common in mobile phones,
therefore this method isless applicable to mobile
solutions.Also the challenge of accurate touchdetection is
even greater when using a single camera and almost any
surface.Floating virtual keyboards, which are portableand
enabledirectingthe camera to any surface, are even more
challengingto implement.The proposed solution is based
on single standard camera; and presents an improved
touch detection method, based on microphone input. Thus
enables working on any surface or paper, as long as both
the camera input and microphone input are there.
The set-up is like this- first the camera is positioned
accordance with the paper which acts as the keyboard and
the microphone which is attached to the paper is interfaced
with the pc with the help of microcontroller. The user has
to apply red nail paint on the finger nails. Now as the user
starts typing in front of the camera and touches the paper,
the sound generated by tap of finger on keyboard is
detected by controller and 1/0 is sent to PC by serial
communication. The Matlab code triggers the camera and
the photo is captured. Then by applying different image
processing operations like segmentation, thresholding etc.
the required character is displayed on the text editor.
Basically image processing is any form of signal
processing for which the input is any form of an image,
such as a photograph or video frame; the output of an
image processing may be either an image or, a set of
characteristics or parameters related to the image. The
Image processing usually refers to digital image
processing, but optical and analog image processing can
also be done. With the help of this proposed system, the
keyboard layout can be generated using a paper. The
image being displayed on the paper is scanned via a
camera connected to the computer.
An image processing application will then detect the key
press events by user and process them accordingly. The
application must also generate virtual key press events for
OS as if a normal keyboard is used. The keyboard layout
can be changed hence allowing user to change the layout
based on application, for example user can select different
languages for editor or a specialized layout for gaming
applications. User can even design his own layout in
hardware version. User must also be able to reconfigure
the keyboard. i.e. the application must allow the user to
map keys based on his preference. This would allow the
user to set the keypad layout according to his
requirements.
There are 3 modules:
Image Capturing
Processing
Display
These modules do the processing when the layout is
provided the module does some processing on the image
captured by the camera. After processing this layout is
stored temporarily. When the user select character from
the keyboard and gives some input, the character is
captured by the camera, processing is done on it, location
of the character is checked. The provided input and the
stored character is compared and the desired output is
displayed on the screen.
Image Segmentation- It is defined as the process of
partitioning a digital image into multiple segments (sets of
pixels, which known as superpixels). The goal of
segmentation is to simplify and/or change the
representation of an image into something that is more
meaningful and easier to analyze. Image segmentation is
typically used to locate objects and boundaries (lines, curves, etc.) in images. More precisely, image
segmentation is the process of assigning a label to every
pixel in an image such that pixels with the same label
share certain characteristics.
The result of image segmentation is a set of segments that
collectively cover the entire image, or a set of contours
extracted from the image. Each of the pixels in a region
are similar with respect to some characteristic or computed
property, such as color, intensity, or texture. Adjacent
regions are significantly different with respect to the same
characteristic(s). When applied to a stack of images,
typical in medical imaging, the resulting contours after
image segmentation can be used to create 3D
reconstructions with the help of interpolation algorithms
like marching cubes.
Thresholding-The simplest method of image segmentation
is called the thresholding method. This method is based on
a clip-level (or a threshold value) to turn a gray-scale
image into a binary image. There is also a balanced
histogram thresholding.The key of this method is to select
the threshold value (or values when multiple-levels are
selected). Several popular methods are used in industry
including the maximum entropy method, Otsu's method
(maximum variance).Recently, methods which have been
developed for thresholding computed tomography (CT)
images. The key point is that, unlike Otsu's method, the
thresholds are derived from the radiographs instead of the
(reconstructed) image.