Abstract: Currently, a growing number of attack sources of distributed denial of service (DDoS) are migrating to cloud computing and bringing a greater security challenge to the whole cyberspace. However, the research on effectively suppressing these attack sources is still deficient. So, this paper proposes a method pTrace to defeat the DDoS attack sources in cloud, which comprising the packet filter module inFilter and the malicious process retroactive module mpTrace. inFilter mainly filters packets with forged source address. And, mpTrace firstly identifies attack streams and their corresponding source addresses, then trace malicious processes based on the obtained source addresses. We have implemented a prototype system under Openstack and Xen environment. Experimental results and analysis show that inFilter can prevent large-scale DDoS attack frombeing launched in cloud center with lower time consumption, and mpTrace can identify a attack flow correctly when its flow rate is about 2.5 times the normal traffic, tracing malicious processes in ms time level. At last, this method reduces the impact both on puppet cloud tenant and the victim outside cloud.

Key words: controllable cloud computing, packets filtering, malicious program tracebacking, information entropy, virtual machine introspection