Abstract: In this paper, nearly perfect reconstruction (NPR) critically sampled cosine modulated M-channel filterbank (CMFB) and transmultiplexer systems are proposed. An improved design approach for a FIR low pass prototype filter that expoits spline function in the transition band of the ideal filter and takes the magnitude squared error function into account is considered. Prototype filter is designed by new Cosh window, which is derived in the same way as Kaiser window but having no power series expansion in its time domain representation and performs better side lobe falloff ratio than the Kaiser window. The procedure involves the univariate unconstrained optimization of an error function. A performance evaluation reveals that redefined error function based design significantly lower distortion and interference parameters. The optimized design approach formulated for the filterbank carries over to the transmultiplexer system.