Can the Multi-Incoming Smart Meter Compressed Streams be Re-Compressed?

Abstract: Smart meters have currently attracted attention because of their high efficiency and throughput performance. They transmit a massive volume of continuously collected waveform readings (e.g. monitoring). Although many compression models are proposed, the unexpected size of these compressed streams required endless storage and management space which poses a unique challenge. Therefore, this paper explores the question of can the compressed smart meter readings be re-compressed? We first investigate the applicability of re-applying general compression algorithms directly on compressed streams. The results were poor due to the lack of redundancy. We further propose a novel technique to enhance the theoretical entropy and exploit that to re-compress. This is successfully achieved by using unsupervised learning as a similarity measurement to cluster the compressed streams into subgroups. The streams in every subgroup have been interleaved, followed by the first derivative to minimize the values and increase the redundancy. After that, two rotation steps have been applied to rearrange the readings in a more consecutive format before applying a developed dynamic run length. Finally, entropy coding is performed. Both mathematical and empirical experiments proved the significant improvement of the compressed streams entropy (i.e. almost reduced by half) and the resultant compression ratio (i.e. up to 50%).