One of the critical issues in big data environment is the volume of the data generated and streamed in real time or archived for later use. If the data streamed in real-time is large enough then the time required to transmit that data may lead to unnecessary delay and latency problems. The other case with huge volumes of data when it is archived is the case of storage devices requirements. Compressing the data before or while transmitting is one useful solution to minimize the size of data and thus avoid or minimize the latency problem and storage shortages. In this research paper the focus will be to deal on some compression techniques, especially in data transforming techniques similar to BWT, MTF and RLE which are commonly used to transform the data prior to encoding the data into fewer bits using entropy encoding techniques such as Huffman, Arithmetic, Golomb, etc. All the data transforming techniques have their own positive and weak sides. Thus in this paper alternative method is proposed to fill some of the gaps that cannot be solved by the already existing data compression transforming techniques. The proposed algorithm can be combined with other data compression methods to optimize the compression efficiency. The performance of this proposed algorithm is measured as compared to other related compression algorithms, and it is found that in some special cases it can perform better than others.

Info-table unique symbol codeword SCIT frequency gap critical issue