Abstract: Heterogeneous computing is a new branch of parallel processing and is expected to reach super speedup. It is an important step for heterogeneous computing to extract the heterogeneity from a program. Such work is very difficult and some concepts and terminologies are ambiguous. From the point of view of program structure and program running, the formal definitions about SIMD and MIMD parallelism are presented clearly, which are criteria of extracting program heterogeneity. Also proposed are two approaches that are based on program structure transformation and performance analysis. Such approaches can serve as the originating framework of automatically extracting the heterogeneity from a program.