Smart HCI with Touch Screen Technologies

The touch screen is going to be a vital component of the human computer interface(HCI) in computing , PDAs , handheld &  wireless devices .New technological developments in touch screens are addressing the demands of the modern way of human interface with computers .Touch Screens provide   interactive display applications and facilitate a more interactive, collaborative environment. ATM machines, kiosks, touch displays, smart phones, computer-based training (CBT), PDAs , notebooks and tablet PCs all make use of this technology. In the future, touch technology will offer users a more symbiotic experience, allowing people to feel as if they are actually handling objects. While such developments are still in progress, there are several important technologies already implemented today. This article also provides the overview of touch screen programming in Visual C++.


Touch technology is a recent technology that first made its appearance in the form of touch screens. Developed by Dr Samuel Hurst in the 1970s, these screens allow users to interact with technology intuitively and easily by touching the screen directly with one finger or by using an input device, typically in the form of a pen. TouchPads are an even more recent technology, having first appeared in notebook computers in 1994. Originally TouchPads intended to provide integrated cursor control in a notebook computer to eliminate the need for an external mouse. TouchPads have proven so popular that USB models for use with workstations have also been introduced.

Touch technology involves detecting, measuring and processing the user's touch input. Today, there are four basic kinds of touch technology: resistive, surface wave, scanning infrared and capacitive.

Each method uses a different method to detect and measure the user’s touch and suitable for different applications.

  • Resistive : this approach responds to pressure on the surface of the screen; a finger, gloved hand, or stylus can be used; suitable for PDAs.
  •  Surface wave : uses ultrasonic waves that pass over touch screen pane; a finger or soft-tipped stylus can be used; suitable for information kiosks .
  • Scanning infrared : makes use of miniature IR LEDs (light-emitting diodes) to create a grid of invisible light; a finger gloved hand or stylus can be used; found in industrial, medical, military and manufacturing equipment.
  • Capacitive : responds to disturbances in an electrical field that are created when the user touches the surface of the screen with a hand or finger.
  • Surface Capacitance requires direct finger contact, a thin latex glove, or special conductive stylus
  • Projected Capacitance allows gloved or direct finger input; used for kiosks and ATMs .

Components of Touch Screens

The touch screen is made up of two main parts:

  • Touch sensor screen
  • Controller

The touch screen is essentially made up of a glass backing panel, which is covered with a thin, metallic resistive film. On top of this resistive coating is a layer of virtually invisible separator dots, which prevent anything from touching the film. Positioned on the separators is a coversheet which is a protective, scratch resistant covering on the outside and another conductive resistive layer on the inside touching the separators. This leaves a space between the two resistive layers. Since the two sheets are conductive, they have a constant electrical current running through them. Pressure applied to the screen will cause the two conductive films to touch resulting in a change in the electrical field at the point of contact. The controller is what connects the touch sensors in the screen and the central processing unit. The voltage is  measured at the point of contact because if the determined voltage is not high enough then the “push” does not register in the controller. If the voltage is of the appropriate level, the controller reads the horizontal location of the point of contact using a grid on the top resistive layer. The vertical location is interpreted in much the same way  however it is found using a grid on the bottom resistive film. This data is then converted to X and Y coordinates and translated into a language that the processor can understand. The processor reads this information in much the same way as mouse commands. The controller is located in the monitor; usually behind the touch screen.

Basic Components

  • Backing panel- Clear glass or acrylic sheet. This is what covers the display screen of the computer monitor or other device. The bottom resistive film lies on top of the backing panel.
  • Resistive film- Thin, metallic layer. There are two resistive films involved in the touch screen; one on top of the backing panel and one on the back of the coversheet. These layers are separated by the separator dots. Since the sheets are conductive, they have a constant electrical current running through them.
  • Separator dots- The separator dots form a barrier between the two resistive films, creating an open space. This space is necessary for the touch screen to function. When the screen is pressed, the two resistive layers will come in contact and create a change in voltage.
  • Coversheet- Clear, protective, scratch-resistant covering. This is what shields all of the delicate resistive layers from the outside environment. Though the coversheet is durable and can guard protect against sunlight, water and prodding, it can easily be severed by sharp objects.
  • Controller- PC card which is the link between the touch screen and the central processing unit. It measures the voltage at the point of contact and then calculates the position of the point in X and Y coordinates. The controller then translates this data into a language that the processor can understand.

Working of Touch Screen

Touch screen works in the coordination of three main components: a touch sensor, a controller and a software driver. To make a complete touch input system, a touch screen is combined with a display and a PC or other device.

1) Touch screen censor
A touch screen sensor is a glass plate having touch responsive surface. The touch sensor is positioned over a display screen so that the responsive area of the plate covers the maximum viewable area of the video screen. There are number of touch sensor technologies available in the market today, each using a different approach to detect touch input. The sensor has an electrical current or signal passing through it and touching the screen causes change in voltage or signal. This voltage or signal change is used to find out the location of the touch to the screen.

2) Controller

The controller used in touch screen is a small PC card that interconnects between the touch sensor and the PC. Controller takes data from the touch sensor and converts it into information that PC can understand. For integrated monitors, the controller is usually installed inside the monitor or it is placed in a plastic case for external touch add-ons/overlays. The controller is useful to determine what type of interface/connection you will need on the PC. Integrated touch monitors are provided with an extra cable connection on the back for the touch screen. Controllers are available by connecting to a Serial/COM port (PC) or to a USB port (PC or Macintosh).

3) Software Driver

The driver is software for the PC system that permits the touch screen and computer to work together. It tells operating system of computer how to interpret the touch event information that is sent from the controller. Today’s touch screen drivers are a mouse-emulation type driver. This makes touching the screen the same as clicking your mouse at the same location on the screen. This permits the touch screen to work with existing software and provide new applications to be developed without the need for touch screen specific programming. Some devices such as thin client terminals, DVD players and specialized computer systems either do not use software drivers or they have their own in -built touch screen driver.
The touch screen is an input device, so it needs to be combined with a display and a PC or other device to make a complete touch input system.

When the user interacts with the Touch Screen, the following processes occur:

  • User's input: The user initiates the touch event by interacting with the sensor grid through the insulating front glass layer.
  • Capacitance: Electrodes on the sensor grid capture information about the amount of capacitance on the x- and y-axes.
  • Measurement: Circuits on the touch controller chip measure the capacitance on each sensor electrode. This is equivalent to determining how hard the finger is pressing on, but it cannot tell where on the electrode the finger is located.
  • Location: The location of the user's finger on the grid is calculated by looking at the distribution and amount of capacitance on all the electrodes. By examining this distribution of capacitance, the touch controller can determine the position of the finger. The points where the largest amount of capacitance on the x-axis and y-axis are mapped together pinpoint the location of the user’s input. By comparing the amount of capacitance on adjacent electrodes (a process referred to as "interpolation"), a more precise location is determined.
  • Processing: The touch controller board processes the information (about the pressure/location of the touch input) and passes it to the computer.
  • System Response: Operating System/Application Software responds to the user's input.

Touch Screen Characteristics

  • Speed: high
  • Accuracy: low (finger), high (pen)
  • Speed control: yes
  • Continuous movement: yes
  • Directness: direction, distance, speed
  • Fatigue: high
  • Footprint: no
  • Best uses: point, select

Touch Screen Pros

  • Direct: Direct pointing to objects, direct relationship between hand and cursor movement (distance, speed and direction), because the hand is moving on the same surface that the cursor is moving, manipulating objects on the screen is similar to manipulating them in the manual world
  • Fast (but less precise without pen)
  • Finger is usable, any pen is usable (usually no cable needed).
  • No keyboard necessary for applications that need menu selections only -> saves desk space
  • Suited to: novices, applications for information retrieval, high-use environments.

Touch Screen Cons

  • Low precision (finger): Imprecise positioning, possible problems with eye parallaxis (with pen, too), the finger may be too large for accurate pointing with small objects -> a pen is more accurate.
  • Hand movements (if used with keyboard): Requires that users move the hand away from the keyboard; a stylus requires also hand movements to take up the pen.
  • Fatigue: Straining the arm muscles under heavy use (especially if the screen is placed vertically).
  • Sitting/Standing position: The user has to sit/stand close to the screen.
  • Dirt: The screen gets dirty from finger prints.
  • Screen coverage: The user's hand, the finger or the pen may obscure parts of the screen.
  • Activation: Usually direct activation of the selected function, when the screen is touched; there is no special "activation" button as with a light pen or a mouse.

Touch Screen Programming

This section provides the overview of creating a touch screen application in Visual C++ & QuickDesigner.
Windows  Messages

  • WM_LBUTTONDOWN, when the user presses the stylus on the screen
  • WM_LBUTTONUP, when the user lifts the stylus
  • WM_MOUSEMOVE, while moving the stylus within the same window while it's down
  • WM_LBUTTONDBLCLK, if the user double taps but only if that window's class was registered with the CS_DBLCLKS style
  • The wParam parameter contains a set of bit flags indicating whether the Ctrl or Shift keys on the keyboard are currently
  • The lParam parameter contains two 16-bit values that indicate the position on the screen of the tap
  • The low-order 16 bits contain the x (horizontal) location relative to the upper left corner of the client area of the window while the high-order 16 bits contain the y (vertical) position Inking
  • A typical application for a handheld device is capturing the user's writing on the screen and storing the result as ink
  • GetMouseMovePoints returns a number of stylus points that didn't result in WM_MOUSEMOVE messages
  • GetMouseMovePoints (PPOINT pptBuf, UINT nB ufPoints, UINT *pnPointsRetrieved); GetMouseMovePoints returns points in the resolution of  the touch panel, not the screen. This is generally set at four times the screen  resolution, so you need to divide the coordinates returned by GetMouseMovePoints by 4 to convert them to screen coordinates

The Touch Screen based TicTac Example Program

// Message dispatch table for MainWindowProc
const struct decodeUINT MainMessages[] = {
WM_CREATE, DoCreateMain,
WM_SIZE, DoSizeMain,
WM_PAINT, DoPaintMain,
WM_DESTROY, DoDestroyMain,

LRESULT DoSizeMain (HWND hWnd, UINT wMsg, WPARAM wParam,
LPARAM lParam) {
RECT rect;
INT i;
// Adjust the size of the client rect to take into account the command bar height.
GetClientRect (hWnd, &rect); += CommandBar_Height (GetDlgItem (hWnd, IDC_CMDBAR));

// Define the playing board rect.
rectBoard = rect;
rectPrompt = rect;
// Layout depends on portrait or landscape screen.
if (rect.right - rect.left > rect.bottom - {
rectBoard.left += 20; += 10;
rectBoard.bottom -= 10; rectBoard.right = rectBoard.bottom - + 10;
rectPrompt.left = rectBoard.right + 10;
} else {
rectBoard.left += 20; rectBoard.right -= 20; += 10; rectBoard.bottom = rectBoard.right - rectBoard.left + 10; = rectBoard.bottom + 10;
return 0;
LRESULT DoLButtonUpMain (HWND hWnd, UINT wMsg, WPARAM wParam,
LPARAM lParam) {
INT cx, cy, nCell = 0;
pt.x = LOWORD (lParam); pt.y = HIWORD (lParam);
// See if pen on board. If so, determine which cell.
if (PtInRect (&rectBoard, pt)){
// Normalize point to upper left corner of board.
pt.x -= rectBoard.left; pt.y -=;
// Compute size of each cell.
cx = (rectBoard.right - rectBoard.left)/3; cy = (rectBoard.bottom -;
nCell = (pt.x / cx); // Find column.
nCell += (pt.y / cy) * 3; // Find row.
if (bBoard[nCell] == 0) { // If cell empty, fill it with mark.
if (bTurn) { bBoard[nCell] = 2;
bTurn = 0;
} else { bBoard[nCell] = 1;
bTurn = 1;
} InvalidateRect (
} else { // Inform the user of the filled cell.
MessageBeep (0);
return 0;
LRESULT DoPaintMain (HWND hWnd, UINT wMsg, WPARAM wParam,
LPARAM lParam) {
RECT rect;
HFONT hFont, hOldFont;
HDC hdc;
// Adjust the size of the client rect to take into account the command bar height.
GetClientRect (hWnd, &rect); += CommandBar_Height (GetDlgItem (hWnd, IDC_CMDBAR));
hdc = BeginPaint (hWnd, &ps);
// Draw the board.
DrawBoard (hdc, &rectBoard);
// Write the prompt to the screen.
hFont = GetStockObject (SYSTEM_FONT);
hOldFont = SelectObject (hdc, hFont);
if (bTurn == 0)
DrawText (hdc, TEXT (" X's turn"), -1, &rectPrompt,
DrawText (hdc, TEXT (" O's turn"), -1, &rectPrompt,
SelectObject (hdc, hOldFont);
EndPaint (hWnd, &ps);


We can develop interactive touch control screen using QuickDesigner, a Windows based editor. The tools provided can be assigned to numeric registers and flags in the  controller for real-time update on your screen. The QuickDesigner software package and software design tools, Bar Graphs, Multi-State Indicators, Numeric Data Entry and Illuminated Push Button Selector Switches, allow the designer to shift the operator interface load from the controller to the QuickPanel Touch Screen, dramatically reducing development time.

Uses of Touch Screen

The touch screen is one of the simplest PC interfaces to use, making it the interface of choice for a large number of applications. Following are uses of touch screen.

  • Public Information Displays

Tourism displays, trade show displays, Information kiosks and other electronic displays are used by large number of people that have little or no computing experience. The touch screen interface is easier to use than other input devices especially for novice users. A touch screen is useful to make your information more easily accessible by allowing users to navigate your presentation by simply touching the display screen.

  • Retail and Restaurant Systems

In retail or restaurant environment, touch screen systems are easy to use so employees can get work done faster and also training time can be reduced for new employees. As input is present right on the screen, valuable counter space can be saved. Touch screens can be used in order entry stations, cash registers, seating, reservation systems and more.

  • Control and Automation Systems

The touch screen device is useful in systems ranging from industrial process control to home automation. Valuable workspace can be saved by integrating the input device with the display. In real-time by simply touching the screen and with a graphical interface, operators can monitor and control complex operations.

  • Computer Based Training

The touch screen interface is more user-friendly than other input devices so overall training time for computer novices and therefore training expense can be reduced. It can also more useful to make learning more fun and interactive, which can lead to a more beneficial training experience for both students and educators.

  • Assistive Technology

The touch screen interface is very useful for those having difficulty using other input devices such as a mouse or keyboard. When touch screen used with software such as on-screen keyboards or other assistive technology, they can help make computing resources more available to people that have difficulty using computers.

Limitations of Touch Screens

Fingers have a certain size. So, screen elements have to have a minimum size, to ensure that a touch screen can be operated with few errors. Even with a stylus, which makes possible to use smaller screen elements, there are limiting factors.

Sequential Input
Input on a touch screen is inherently sequential: One finger is used for clicking. This slows input down compared to keyboard input where several fingers can be used virtually in parallel.

Keying in many numbers or letters by pointing with the finger is also very straining and tiring. Therefore, touch screens make no sense in workplaces, where much text or number input is required.

On touch screens, there is no analogue to mouse-move events. Mouse users can move the mouse pointer over screen elements, get feedback about the selected element (e.g. by highlighting), and may confirm the selection by clicking the mouse button. Touch screen users directly point on a screen element. If they are lucky, they can withdraw their finger if they touched the wrong screen element. On other touch screens, the touch immediately initiates an action - there is no opportunity to cancel the action.

Drag Operations
Dragging is generally not well suited to finger-operated touch screens; here pointing is the preferred interaction. However, this is different for stylus-operated touch screens. Here gestures and handwriting offer promising possibilities for making interaction with computers easier and more intuitive. But here, too is the limitation of strictly sequential input. There are also no means to constrain drag operations to, e.g. straight lines, like with mouse-based interfaces.

Future of Touch Sensor Technology

Touch sensor technology has made huge advancements in recent years. One of the most impacting developments is the invention of “multi touch” technology by Fingerworks. With multi touch sensors, one can use multiple fingers to move or select different objects on a screen. This equipment was originally created for use on a large screen for teaching purposes however it is being applied to many new gadgets. One in particular is the brand new iPhone. The iPhone features multi touch technology, which allows everything to be done on the screen of the phone with multiple fingers. One can zoom in and out of pictures by simply pinching the screen with two fingers or moving them apart. When the iPhone is revealed in June of 2007, it will introduce multi-touch technology to the public.


Touch screens have been growing more and more popular recently due to their straightforwardness and low cost. The main benefit of this technology is the fact that it does not require the use of a mouse. One simply needs to touch his or her finger to the screen to issue a command. This is advantageous because mice can be easily stolen or damaged if the monitor is for public use such as those used in stores for credit cards. The lifespan for one screen is approximately 3 million touches.

For more details you can refer the following references or e-mail at :

References & Web Links

  • Holsinger, Erik. “How Multimedia Works”  Ziff-Davis Press.
  •  Mayhew, D. J. “ Principles and guidelines in software user interface design. ” Prentice Hall.
  • Shneiderman, B. “ Designing the user interface: Strategies for effective human-computer interaction. “ Addison-Wesley.