Residential College | false |
Status | 已發表Published |
Toward fast theta‐join: A prefiltering and amalgamated partitioning approach | |
Jiashu Wu1,2; Yang Wang1; Xiaopeng Fan1; Kejiang Ye1; Chengzhong Xu3 | |
2022-04-11 | |
Source Publication | Concurrency and Computation: Practice and Experience |
ISSN | 1532-0626 |
Volume | 34Issue:17Pages:e6996 |
Abstract | As one of the most useful online processing techniques, the theta-join operation has been utilized by many applications to fully excavate the relationships between data streams in various scenarios. As such, constant research efforts have been put to optimize its performance in the distributed environment, which is typically characterized by reducing the number of Cartesian products as much as possible. In this article, we design and implement a novel fast theta-join algorithm, called Prefap, by developing two distinct techniques—prefiltering and amalgamated partitioning—based on the state-of-the-art FastThetaJoin algorithm to optimize the efficiency of the theta-join operation. Firstly, we develop a prefiltering strategy before data streams are partitioned to reduce the amount of data to be involved and benefit a more fine-grained partitioning. Secondly, to avoid the data streams being partitioned in a coarse-grained isolated manner and improve the quality of the partition-level filtering, we introduce an amalgamated partitioning mechanism that can amalgamate the partitioning boundaries of two data streams to assist a fine-grained partitioning. With the integration of these two techniques into the existing FastThetaJoin algorithm, we design and implement a new framework to achieve a decreased number of Cartesian products and a higher theta-join efficiency. By comparing with existing algorithms, FastThetaJoin in particular, we evaluate the performance of Prefap on both synthetic and real data streams from two-way to multiway theta-join to demonstrate its superiority. |
Keyword | Amalgamated Data Stream Partitioning Cartesian Product Reduction Online Data Stream Prefiltering Theta-join (Θ-join) |
DOI | 10.1002/cpe.6996 |
URL | View the original |
Indexed By | SCIE |
Language | 英語English |
WOS Research Area | Computer Science |
WOS Subject | Computer Science, Software Engineering ; Computer Science, Theory & Methods |
WOS ID | WOS:000779988000001 |
Publisher | WILEY, 111 RIVER ST, HOBOKEN 07030-5774, NJ |
Scopus ID | 2-s2.0-85127802148 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | Faculty of Science and Technology |
Corresponding Author | Xiaopeng Fan |
Affiliation | 1.Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China 2.University of Chinese Academy of Sciences, Beijing, China 3.University of Macau |
Recommended Citation GB/T 7714 | Jiashu Wu,Yang Wang,Xiaopeng Fan,et al. Toward fast theta‐join: A prefiltering and amalgamated partitioning approach[J]. Concurrency and Computation: Practice and Experience, 2022, 34(17), e6996. |
APA | Jiashu Wu., Yang Wang., Xiaopeng Fan., Kejiang Ye., & Chengzhong Xu (2022). Toward fast theta‐join: A prefiltering and amalgamated partitioning approach. Concurrency and Computation: Practice and Experience, 34(17), e6996. |
MLA | Jiashu Wu,et al."Toward fast theta‐join: A prefiltering and amalgamated partitioning approach".Concurrency and Computation: Practice and Experience 34.17(2022):e6996. |
Files in This Item: | Download All | |||||
File Name/Size | Publications | Version | Access | License | ||
2022_Toward fast the(2200KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | View Download |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment