Abstract: A feature extraction method is proposed for discriminative features on segmented moving objects and these compact set of color feature for object representation tend to handle the large amount of local features in feature correspondence and high accuracy in object classification in challenging sequences. Test sequences from the Caviar Test case scenario dataset and online benchmark tracking dataset are used in experiment to prove the robustness of proposed method.