Voice over IP is becoming increasingly a valid alternative to traditional telephony networks because of its low cost, higher flexibility and easy integration. On the other hand Spam over IP Telephony (SPIT) is increasing because of the same vey benefits. There have been numerous proposals in regard to prevention of SPIT however it is very difficult to place a call under the category of SPAM or not on the basis of computations since it totally relies on the uneasiness caused to the end user. Black Listing and Grey Listing are well known methods in SPIT prevention which is based on analyzing call detail records. This paper proposes a method of controlling the SPAM centrally through a server based on Grey Listing and Black Listing recording SPAM flag based on the digits entered by end user on the termination or during the call.