Abstract: In this paper an accelerometer based portable device for digit and gesture recognition is being presented. The recognized data will be displayed on the monitor. A speech module is also embedded. The system finds its application in assisting patients. Apart from its HCI and gaming applications, it can be incorporated to assist the communication of deaf and dump with the visually challenged people. The digital pen consists of a tri-axial accelerometer, a microcontroller and a display unit. The system operates with microcontroller as its heart. The system is capable of identifying the digits written on air, display it on PC and also give a voice corresponding to the digit written. Users can use the pen to write digits or make hand gestures, and the accelerations of hand motions measured by the accelerometer are transmitted to a computer for online trajectory recognition. The digital pen can also control the operation of other devices. Here a fan and light are made to turn on and off with the aid of gestures made by the digital pen. As a future application more devices can be controlled with its aid.