This paper proposes a monitoring system called GUESS to compare the performance and energy consumption of join query processing on Spark in serverless and serverful environments. The system collects metrics on resource utilization, query execution times, and power usage through Prometheus, Grafana, Spark History Server, and OpenManage Enterprise Power Manager. These metrics are visualized through an intuitive web dashboard to easily compare the serverless and serverful Spark workloads. Experimental results using the TPC-H benchmark show that the serverless environment consumes less energy than the serverful environment due to on-demand resource allocation. However, the serverful environment exhibits better query performance, especially for workloads with known resource requirements. GUESS provides insights into optimizing resource efficiency and query performance when deploying Spark analytic workloads.
Tạp chí khoa học Trường Đại học Cần Thơ
Lầu 4, Nhà Điều Hành, Khu II, đường 3/2, P. Xuân Khánh, Q. Ninh Kiều, TP. Cần Thơ
Điện thoại: (0292) 3 872 157; Email: tapchidhct@ctu.edu.vn
Chương trình chạy tốt nhất trên trình duyệt IE 9+ & FF 16+, độ phân giải màn hình 1024x768 trở lên