Integrity checking of operating systems with respect to kernel level malware

Abstract

Kernel-mode rootkits represent a considerable threat to any computer system, as they provide an intruder with the ability to hide the presence of his malicious activity. These rootkits make changes to the operating system s kernel, thereby providing particularly stealthy hiding techniques. This thesis addresses the problem of collecting reliable information from a system compromised by kernel-mode rootkits. It looks at the possibility of using virtualization as a means to facilitate kernel-mode rootkit detection through integrity checking. It describes several areas within the Linux kernel, which are commonly subverted by kernel-mode rootkits. Further, it introduces the reader to the concept of virtualization, before the kernel-mode rootkit threat is addressed through a description of their hiding methodologies. Some of the existing methods for malware detection are also described and analysed. A number of general requirements, which need to be satisfied by a general model enabling kernel-mode rootkit detection, are identified. A model addressing these requirements is suggested, and a framework implementing the model is set-up.