Abstract: In applications such as online games, online data services and wireless sensor networks, there is a need for large-scale servers to handle high volume of soft real-time requests. Such clusters must adaptively control the CPU utilization of heterogeneous processors in order to maintain desired performance and prevent system overload in face of unpredictable workloads. According to this, the authors propose a new method named heterogeneous distributed utilization control with load balancing (HDUC-LB) to handle unpredictable workload and improve system QoS in heterogeneous real-time clusters. HDUC-LB mainly includes two parts. First, it presents a new load balancing model for heterogeneous real-time clusters to handle overload of the system. Then, a utilization control method which is based on feed-back theory is proposed to enforce every desired utilization bounds of each processor. Simulation results show that HDUC-LB can provide a robust utilization control and ensure load balancing between different processors in the open and unpredictable environment, and both the load balancing model and the utilization control method can work effectively and robustly in large-scale clusters.