Software Integrity Verification Based on VMM-Level System Call Analysis Technique

In virtualized cloud computing platform, the key security problem is to guarantee trustworthiness of the softwares which are running in the platform. Integrity measurement and verification has been proposed and studied by many researchers as an effective way to verify the integrity of computer systems. However, existing approaches have some limitations on compatibility, security and maintainability, and cannot be applied into the cloud computing platform. In this paper, we propose a approach named VMGuard, which leverages VMM to enable take integrity measurement outside the operating system. We adopt VMM-based system call interception technique to detect the execution of binaries. System call correlation and guest OS file system metadata reconstruction are proposed to verify the integrity of software in guest OS. We have developed a prototype of VMGuard and implemented it in two mainstream virtual machine monitors, Qemu and KVM, respectively. We also evaluate the effectiveness and performance overhead of our approach by comprehensive experiments. The results show that VMGuard achieves effective integrity measurement with less than 10% overhead.