Detecting and stopping privacy violations in software

Privacy violations caused by software that steals sensitive information from a user's computer have become a significant security threat in a short period of time. Unfortunately, previous work that address privacy violations have shortcomings because they rely on either information-flow control or signature scanning. Information-flow-based systems either require source code or are extremely slow and therefore are impractical to use as real-time detection tools. Signature-scanning-based systems rely on vendors to provide signature updates and therefore are ineffective against previously unseen software.In this dissertation, I show that privacy violations caused by software can be effectively detected and stopped by non-information-flow-based mechanisms, such as anomaly detection and sandboxing. I present two systems, NetSpy and KernelSpy, that I have developed and demonstrate that they are both effective and practical in detecting and stopping privacy violations.The first system, NetSpy, detects the presence of privacy-violating software by applying anomaly detection techniques to identify out-bound network traffic generated by the spying software. NetSpy then stops a spyware program from leaking sensitive data by automatically generating a signature for the spyware using the identified network traffic produced by the spyware. This signature can be automatically pushed to a network intrusion detection system (NIDS), which can use the newly created signature to protect all other computers on the same network. Therefore, NetSpy can not only detect previously unseen spyware, but also reduce the reliance on vendors for timely signature updates. The second system, KemelSpy, is a kernel-level sandbox that can monitor and control run-time kernel activities, such as file system, network, and process, generated by each process running on a computer. KernelSpy intercepts a process' run-time activities and checks them against a set of security policies, which describe undesirable activities that should be stopped. If a process violates a user's privacy according to the security policies, necessary actions, also specified by the policies, are carried out by KernelSpy to stop the offending activities. KernelSpy has low run-time overhead and can be used as a real-time detection tool. Experiments demonstrate that KernelSpy is effective in detecting and preventing privacy violations caused by real-world spyware.