Abstract: In pattern recognition and machine learning, similarity plays an important role. It is well known that similarity has different definitions. Discussed in this paper are different explanations of similarity under exemplar theory and prototype theory. As for exemplar theory, similarity is defined as pair similarity between two objects; and for prototype theory, similarity is defined between an object and a prototype. According to the above analysis, it is pointed out that almost nonnegative physical measures have their interpretations of similarity and similarity measure reflects some global properties in some sense. For instance, similarity can offer a new interpretation for image and fuzzy set. More importantly, also introduced is a new interpretation of Wertheimers contrast invariant principle from similarity point of view. When using similarity, a binary decision: yes or no, is often made. Therefore, it is very interesting to get a binary representation of similarity. A mathematical model between similarity and a binary variable is established by using Taylor expansion. Based on such a result, the binary representation of nonnegative bounded matrix is presented, which leads to the optimal binary decomposition of the similarity matrix. As many applications use similarity matrix, such model is potentially useful.