高级检索

    DHMC:一种有效的高维Cube并行分布式存储结构

    DHMC:An Improved Parallel & Distributed Storage Structure for High-Dimensional Cube

    • 摘要: 在数据仓库系统中,数据立方体(Cube)及其预聚集处理在OLAP起到非常重要的作用.对于一个d维的data Cube可以生成2\+d个聚集Cuboids和∏di=1(|D\-i|+1)个聚集数据单元,但对于一个高维Cube,要创建这些所有聚集Cuboids是不现实的.提出通过共享分段立方体Mini-Cube的高维Cube并行分布式存储结构(DHMC),将高维Cube划分成若干个低维共享分段立方体Mini-Cube,利用并行分布式处理技术来创建这些分割的分段共享Mini-Cube及其聚集Cuboids,来实现高维Cube的并行创建和增量更新维护,从而解决高维OLAP聚集海量数据的存储与查询问题.理论分析与实验结果均表明DHMC性能最佳.

       

      Abstract: Data cube and its pre-computation have been playing an essential role in fast OLAP (online analytical processing) in many data warehouses. For the cube with d dimensions, it can generate 2\+d cuboids and ∏di=1(|D\-i|+1) aggregate cells. But in a high-dimensional cube, it might not be practical to build all these cuboids. In this paper, a novel parallel and distributed storage structure is proposed for high-dimensional cube based on shell segment mini-cubes (DHMC). DHMC partitions the high dimensional cube into some low-dimensional shell segment mini-cubes. OLAP queries are computed online by dynamically constructing cuboids from these shell segment mini-cubes through the parallel & distributed processing system. With this design, for high-dimensional OLAP, the total space that needs to store such shell segment mini-cubes is negligible in comparison with a high-dimensional cube. Such an approach permits a significant reduction of CPU and I/O overhead for many queries by restricting the number of cube segments to be processed for both the fact table and bitmap indices. The proposed data allocation and processing model supports parallel I/O and parallel processing, as well as load balancing for disks and processors. The methods of shell mini-cube are compared with other existing ones such as full cube and partial cube. The analytical and experimental results show that the algorithms of DHMC proposed are more efficient than the other existing ones.

       

    /

    返回文章
    返回