Due to the inconsistent user preferences and the inconsistent rating criteria, the ratings given by different users to one service are actually incomparable, and the reputation mechanism based on assumption of the consistent rating criteria cannot guarantee the comparability among different service reputations, which will result in unobjective outcome when the reputations are used to choose services. To improve the objectivity of online services reputation measurement under the circumstance referred above, this paper presents a method of online service reputation measurement based on Kendall tau distance. Firstly, a distance metric is defined to measure the consistency between the two rating vectors. Secondly, the measurement of online service reputation is modeled as an optimization problem to find a reputation vector that minimizes the Kendall tau distance between the reputation vector and the user-service rating matrix. Finally, simulated annealing algorithm is used to solve the optimization problem and the reputation vector is served as a service reputation. The rationality and effectiveness of the method have been verified by experimental study. The experiments show that the method can meet the preferences of most users, so that users can make right services choice decision, and ensure the efficiency while improving the manipulation resistance ability of the reputation measurement method.