MapReduce has become an attractive and dominant model for processing large-scale datasets. However, this model is not designed to directly support operations with multiple inputs as joins. Many studies on join algorithms including Bloom join in MapReduce have been conducted but they still have too much non-joining data generated and transmitted over the network. This research will help us eliminate the problem by providing an intersection filter based on probabilistic models to remove most disjoint elements between two datasets. Namely, three ways are proposed to build the intersection Bloom filter. To apply the filter to joins, a corresponding MapReduce job will be adjusted in a consistent way without increasing related costs. We then consider two-way joins and join cascades and analyze their costs. As a result, thanks to the high accuracy intersection filter, join processing can minimize disk I/O and communication costs. Finally, the research is proved to be more effective than existing solutions through a cost-based comparison of joins using different approaches.
Tạp chí: La motivation dans l'apprentissage et l'enseignement du FLE - Centre Regional Francophone pour l'EUrope Centrale et Oriental (CREFECO), Sofia, 23-25/10/2013
Tạp chí khoa học Trường Đại học Cần Thơ
Lầu 4, Nhà Điều Hành, Khu II, đường 3/2, P. Xuân Khánh, Q. Ninh Kiều, TP. Cần Thơ
Điện thoại: (0292) 3 872 157; Email: tapchidhct@ctu.edu.vn
Chương trình chạy tốt nhất trên trình duyệt IE 9+ & FF 16+, độ phân giải màn hình 1024x768 trở lên