Abstract: Accelerometer is one of the prominent sensors which are commonly embedded in new age handheld devices. Accelerometer measures acceleration forces in three orthogonal axes X, Y, Z. The raw acceleration values obtained due to the movement of device in 3D space which is hosting accelerometer can be used to interact and control wide range of applications running on the device and can also be integrated with desktop applications to enable intuitive ways of interaction. The goal of the project is to build a generic and economic, gesture recognition framework based on accelerometer sensor and enable seamless integration with desktop applications by providing natural ways of interaction with desktop applications based on the gesture information obtained from accelerometer sensor embedded in Smartphone device held in user hand.