Mutual Information of a class of Poisson-type Channels using Markov Renewal Theory
Abstract: The mutual information (MI) of Poisson-type channels has been linked to a filtering problem since the 70s, but its evaluation for specific continuous-time, discrete-state systems remains a demanding task. As an advantage, Markov renewal processes (MrP) retain their renewal property under state space filtering. This offers a way to solve the filtering problem analytically for small systems. We consider a class of communication systems $X \to Y$ that can be derived from an MrP by a custom filtering procedure. For the subclasses, where (i) $Y$ is a renewal process or (ii) $(X,Y)$ belongs to a class of MrPs, we provide an evolution equation for finite transmission duration $T>0$ and limit theorems for $T \to \infty$ that facilitate simulation-free evaluation of the MI $\mathbb{I}(X_{[0,T]}; Y_{[0,T]})$ and its associated mutual information rate (MIR). In other cases, simulation cost is reduced to the marginal system $(X,Y)$ or $Y$. We show that systems with an additional $X$-modulating level $C$, which statically chooses between different processes $X_{[0,T]}(c)$, can naturally be included in our framework, thereby giving an expression for $\mathbb{I}(C; Y_{[0,T]})$. Our primary contribution is to apply the results of classical (Markov renewal) filtering theory in a novel manner to the problem of exactly computing the MI/MIR. The theoretical framework is showcased in an application to bacterial gene expression, where filtering is analytically tractable.
- I. Bar-David, “Communication under the Poisson regime,” IEEE Transactions on Information Theory, vol. 15, no. 1, pp. 31–37, 1969.
- O. Macchi and B. Picinbono, “Estimation and detection of weak optical signals,” IEEE Transactions on Information Theory, vol. 18, no. 5, pp. 562–573, 1972.
- P. J. Thomas and A. W. Eckford, “Capacity of a Simple Intercellular Signal Transduction Channel,” IEEE Transactions on information Theory, vol. 62, no. 12, pp. 7358–7382, 2016.
- M. Sinzger, M. Gehri, and H. Koeppl, “Poisson channel with binary Markov input and average sojourn time constraint,” in 2020 IEEE International Symposium on Information Theory (ISIT). IEEE, 2020, pp. 2873–2878.
- J. Cannon, “Analytical Calculation of Mutual Information between Weakly Coupled Poisson-Spiking Neurons in Models of Dynamically Gated Communication,” Neural Computation, vol. 29, no. 1, pp. 118–145, 2016.
- M. Behar and A. Hoffmann, “Understanding the temporal codes of intra-cellular signals,” Current opinion in genetics & development, vol. 20, no. 6, pp. 684–693, 2010.
- J. E. Purvis and G. Lahav, “Encoding and Decoding Cellular Information through Signaling Dynamics,” Cell, vol. 152, no. 5, pp. 945–956, 2013.
- D. Friedrich, L. Friedel, A. Finzel, A. Herrmann, S. Preibisch, and A. Loewer, “Stochastic transcription in the p53-mediated response to DNA damage is modulated by burst frequency,” Molecular Systems Biology, vol. 15, no. 12, p. e9068, 2019.
- F. Tostevin and P. R. Ten Wolde, “Mutual Information between Input and Output Trajectories of Biochemical Networks,” Physical review letters, vol. 102, no. 21, p. 218101, 2009.
- I. Lestas, G. Vinnicombe, and J. Paulsson, “Fundamental limits on the suppression of molecular fluctuations,” Nature, vol. 467, no. 7312, pp. 174–178, 2010.
- Y. Nakahira, F. Xiao, V. Kostina, and J. C. Doyle, “Fundamental limits and achievable performance in biomolecular control,” in 2018 Annual American Control Conference (ACC). IEEE, 2018, pp. 2707–2714.
- L. Duso and C. Zechner, “Path mutual information for a class of biochemical reaction networks,” in 2019 IEEE 58th Conference on Decision and Control (CDC). IEEE, 2019, pp. 6610–6615.
- M. Reinhardt, G. Tkačik, and P. R. Ten Wolde, “Path Weight Sampling: Exact Monte Carlo Computation of the Mutual Information between Stochastic Trajectories,” Physical Review X, vol. 13, no. 4, p. 041017, 2023.
- M. Sinzger-D’Angelo and H. Koeppl, “Counting Processes with Piecewise-Deterministic Markov Conditional Intensity: Asymptotic Analysis, Implementation, and Information-Theoretic Use,” IEEE Transactions on Information Theory, 2023.
- A.-L. Moor and C. Zechner, “Dynamic information transfer in stochastic biochemical networks,” Physical Review Research, vol. 5, no. 1, p. 013032, 2023.
- J. Selimkhanov, B. Taylor, J. Yao, A. Pilko, J. Albeck, A. Hoffmann, L. Tsimring, and R. Wollman, “Accurate information transmission through dynamic biochemical signaling networks,” Science, vol. 346, no. 6215, pp. 1370–1373, 2014.
- S. A. Cepeda-Humerez, J. Ruess, and G. Tkačik, “Estimating information in time-varying signals,” PLoS computational biology, vol. 15, no. 9, p. e1007290, 2019.
- Y. Tang, A. Adelaja, F. X.-F. Ye, E. Deeds, R. Wollman, and A. Hoffmann, “Quantifying information accumulation encoded in the dynamics of biochemical signaling,” Nature communications, vol. 12, no. 1, pp. 1–10, 2021.
- P. M. Brémaud, “A Martingale Approach to Point-Processes,” Ph.D. dissertation, EECS Department, University of California, Berkeley, Aug 1972.
- B. Grigelionis, “Mutual information for locally infinitely divisible random processes,” Lithuanian mathematical transactions of the Academy of Sciences of the Lithuanian SSR, vol. 14, no. 1, pp. 1–6, 1974.
- R. Boel, P. Varaiya, and E. Wong, “Martingales on jump processes. II: Applications,” SIAM Journal on Control, vol. 13, no. 5, pp. 1022–1061, 1975.
- J. Massey et al., “Causality, feedback and directed information,” in Proc. Int. Symp. Inf. Theory Applic.(ISITA-90), 1990, pp. 303–305.
- Y. M. Kabanov, “The capacity of a channel of the Poisson type,” Theory of Probability & Its Applications, vol. 23, no. 1, pp. 143–147, 1978.
- E. Çinlar, “Markov Renewal Theory,” Advances in Applied Probability, vol. 1, no. 2, pp. 123–187, 1969.
- A. Häkkinen and A. S. Ribeiro, “Evolving kinetics of gene expression in stochastic environments,” Computational Biology and Chemistry, vol. 37, pp. 11–16, Apr. 2012.
- A. Sanchez, H. G. Garcia, D. Jones, R. Phillips, and J. Kondev, “Effect of Promoter Architecture on the Cell-to-Cell Variability in Gene Expression,” PLOS Computational Biology, vol. 7, no. 3, p. e1001100, Mar. 2011, publisher: Public Library of Science.
- M. Ali Al-Radhawi, D. Del Vecchio, and E. D. Sontag, “Multi-modality in gene regulatory networks with slow promoter kinetics,” PLOS Computational Biology, vol. 15, no. 2, p. e1006784, Feb. 2019.
- M. Davis, “Capacity and cutoff rate for Poisson-type channels,” IEEE Transactions on Information Theory, vol. 26, no. 6, pp. 710–715, 1980.
- H. Hassanzadeh and M. Pooladi-Darvish, “Comparison of different numerical Laplace inversion methods for engineering applications,” Applied mathematics and computation, vol. 189, no. 2, pp. 1966–1981, 2007.
- W. R. Nunn and A. M. Desiderio, “Semi-Markov Processes: An Introduction,” Cent. Nav. Anal, pp. 1–30, 1977.
- E. Çinlar, “Periodicity in Markov renewal theory,” Advances in Applied Probability, vol. 6, no. 1, pp. 61–78, 1974.
- W. L. Smith, “On necessary and sufficient conditions for the convergence of the renewal density,” Transactions of the American Mathematical Society, vol. 104, no. 1, pp. 79–100, 1962.
- W. Rudin, “Real and complex analysis,” New York [u.a.], 1987.
- A. Häkkinen and A. S. Ribeiro, “Characterizing rate limiting steps in transcription from RNA production times in live cells,” Bioinformatics, vol. 32, no. 9, pp. 1346–1352, 2016.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.