DOOC: A Software/Hardware Co-managed Cache Architecture for Reducing Cache Thrashing

Cache memory has become an essential part of modern processors to bridge the increasing gap between the CPU and the main memory speeds. Cache thrashing is one of the most severe problems that affect cache performance. Research shows that the traditional unified cache management by the hardware does not match the diverse memory access patterns in programs. This consequently leads to an enormous cache thrashing. The authors present data-object oriented cache (DOOC), a novel software/hardware cooperative cache. DOOC dynamically allocates isolated cache segments for different data-objects. Moreover, the segment capacity, associativity, block size, and cache coherence protocol implementation of each segment can be configured dynamically by the software. DOOC uses varied cache segments to match diverse data access patterns. The design and implementation of DOOC are discussed in detail together with the compiling techniques and data pre-fetching strategies. Also estimated is the hardware cost of DOOC with CACTI and a sample implementation on LEON3 processor through FPGA. The results show that the DOOC is hardware efficient. Likewise, the performance of DOOC is tested on both single-core and multi-core platforms through software simulation. Fifteen kernel benchmarks extracted from scientific applications, multimedia programs, and database management routines are tested on a single-core platform. Compared with a traditional cache, the DOOC achieves an average reduction of 44.98% in miss rate (the maximum is 93.02%) and an average speedup of 1.20 (the maximum is 2.36). Then the OpenMP version of NPB is run on a four-core platform and the results show an average reduction of 49.69% in miss rate (the maximum is 73.99%).