Operating systems play a central role in securing most of today's systems. In particular, cloud services often rely on the OS to enforce isolation between tenants. For instance, containers are increasingly used in PaaS clouds, and tenant isolation is therefore hinged on the trustworthiness of the OS kernel.
We explore novel mechanisms that improve the security of the OS kernel in a practical and efficient way. Examples of such techniques include kernel attack surface reduction, kernel hardening, as well as static and dynamic analysis techniques to find vulnerabilities in OS kernels.
Split Kernel makes kernel hardening more practical by allowing the enablement and disablement of run-time kernel modifications at process (or kernel-thread) granularity. This enables the development of kernel self-protection mechanisms that may have been previously disregarded for being unlikely to be efficient and practical.
We have implemented three kernel hardening mechanisms through assembly rewriting: stack exhaustion prevention, stack clearance to prevent uninitialized stack variable-related vulnerabilities, and kernel function pointer protection.
For instance, when Split Kernel is not used, these hardening mechanisms impose a 33% overhead on SSH connection times. This performance overhead would be considered too high for practical deployment. With Split Kernel, we can choose to enable these kernel hardening mechanisms only when the SSH daemon is running sandboxed. In that case, the overhead is reduced to less than 1%, which is negligible and makes the use of these hardening mechanisms practical.
Our CCS 2014 paper contains many more use cases, as well as an explanation of the overall design, technical details of the implementation, and security considerations related to the use of Split Kernel (in most cases, the performance benefits come without any security drawbacks).
Overview of the Split Kernel design.