The price of ignorance: The impact of side-information on delay for lossless source-coding

Abstract: Inspired by the context of compressing encrypted sources, this paper considers the general tradeoff between rate, end-to-end delay, and probability of error for lossless source coding with side-information. The notion of end-to-end delay is made precise by considering a sequential setting in which source symbols are revealed in real time and need to be reconstructed at the decoder within a certain fixed latency requirement. Upper bounds are derived on the reliability functions with delay when side-information is known only to the decoder as well as when it is also known at the encoder. When the encoder is not ignorant of the side-information (including the trivial case when there is no side-information), it is possible to have substantially better tradeoffs between delay and probability of error at all rates. This shows that there is a fundamental price of ignorance in terms of end-to-end delay when the encoder is not aware of the side information. This effect is not visible if only fixed-block-length codes are considered. In this way, side-information in source-coding plays a role analogous to that of feedback in channel coding. While the theorems in this paper are asymptotic in terms of long delays and low probabilities of error, an example is used to show that the qualitative effects described here are significant even at short and moderate delays.