Abstract:
The diffusion models have made significant achievements in the field of image generation, but they still face some challenges such as high computational cost, long inference latency, and poor image generation quality during image generation. Based on previous research, this study proposes an efficient adaptive diffusion model inference method based on the warmup-aware and noise reuse methods, which can optimize the performance of the diffusion models according to input conditions and solve the problems of slow inference speed and poor image generation quality. Specifically, our warmup-aware method can dynamically adjust the number of warmup time steps in the initial stage of the diffusion process according to the semantic complexity of the input text prompts to ensure efficient initialization. Subsequently, our noise reuse method dynamically determines the number of time steps for noise reuse in the iterative refinement stage, and reduces unnecessary computational overhead. As shown in Figure 1, We achieved 2.66 times acceleration on 4 NVIDIA V100 GPUs, and compared with DistriFusion, our inference speed increased by 12.8% and the image generation quality improved by 5.03 times. These improvements indicate that our proposed method has significant effects in balancing inference efficiency and image generation quality, and provides a scalable solution for real-time image generation applications.