Kernel Refcount Bug Detection Based on the Consistency of Error Path Behavior

Reference counting (refcount) bugs in the kernel could cause critical security problems including memory leak and use-after-free vulnerabilities. To detect such defects, we propose a refcount bug detection system based on consistency analysis of error path behavior. Compared with the existing work, our method introduces semantic information of the error paths to infer the appropriate refcount behavior on these paths, thus detecting refcount defects cannot be covered by the existing work. First, the system identifies all the error paths in the target function based on the function return value and fault handling code. Second, path-sensitive analysis is performed to collect the specific refcount behavior on each error path within the target function, which is aggregated to infer the dominant tendency of refcount behavior of the error paths in the target function. Finally, based on the idea of consistency checking, the error paths whose refcount behavior is inconsistent with the dominant tendency are identified as potential refcount bugs. In the evaluation, the proposed system finds 21 and 9 bugs on Linux kernel version 5.6-rc2 and version 5.17, respectively, most of which have been confirmed by the kernel developers. In addition, on kernel version 5.6-rc2, the system detects 9 new refcount bugs that could not be identified by existing work.