Abstract: Federated learning (FL) enables multiple clients to train models collaboratively without centralizing raw data, and has been widely studied for privacy-sensitive applications such as healthcare, finance, edge intelligence, and smart cities. In realistic deployments, however, clients differ not only in data distributions, but also in system resources, model interfaces, optimization objectives, and privacy or governance constraints. These heterogeneities reshape federated optimization and directly affect convergence stability, communication efficiency, and model generalization. To reveal how heterogeneity affects federated optimization, this survey develops a source-oriented taxonomy of heterogeneous FL that covers three primary dimensions, namely data heterogeneity, system heterogeneity, and modeling heterogeneity, and further refines them into 14 co-occurring second-level types with explicit decision criteria. Beyond extending coverage, the taxonomy links heterogeneous sources to four optimization-impact channels: objective mismatch, participation bias, update distortion, and collaboration-interface misalignment. Based on this framework, we synthesize representative methods and compare their assumptions, communication and computation costs, and convergence guarantees, distilling technical evolution patterns, identifying key research bottlenecks, and offering researchers a structured technical reference. We further distill practical guidelines for mechanism selection under many different combinations of heterogeneity sources. Finally, we discuss multi-heterogeneity coupling, benchmark construction, engineering platforms, and heterogeneous FL in the era of foundation models.