Abstract: In the environment of Xen virtual machines, Credit algorithm is non-preemptible. When virtual CPU is idle, it can not inform Xen of its idle state, which leads to its keeping the usage of physical CPU. Currently, the existing methods to process idle virtual CPU only paid attention to the optimization of guest OS, and neglected that of service OS. That is to say, it has three problems to optimize, such as the waste of the time of idle virtual CPU, no consideration of idle state of service OS and the inaccuracy of the judgement of virtual machine's idle state, which causes great much needless loss. According to the problems, an idle virtual CPU scheduling algorithm based on Credit algorithm is given. When the message of virtual CPU's idle state is notified by reception module of idle state of virtual CPU, virtual CPU's credit of virtual machine is modified dynamically, in order to allocate the reminder of idle time to other virtual machine. At the same time, the weight of virtual machine is also modified according to the virtual CPUs' average idle rate of the virtual machine, which realizes the dynamic integration between virtual machine's scheduling method and feedback control. The experiment results show that the idle virtual CPU scheduling algorithm improves greatly the performance of virtual machines.