Directed Redundancy in Time Series
Abstract: We quantify the average amount of redundant information that is transferred from a subset of relevant random source processes to a target process. To identify the relevant source processes, we consider those that are connected to the target process and in addition share a certain proportion of the total information causally provided to the target. Even if the relevant processes have no directed information exchange between them, they can still causally provide redundant information to the target. This makes it difficult to identify the relevant processes. To solve this issue, we propose the existence of a hidden redundancy process that governs the shared information among the relevant processes. We bound the redundancy by the minimal average directed redundancy from the relevant processes to the target, from the hidden redundancy process to the target, and from the hidden redundancy process to the relevant processes.
- R. Yeung, “A first course in information theory,” Springer, 2002.
- P. L. Williams and R. D. Beer, “Nonnegative decomposition of multivariate information,” CoRR, vol. abs/1004.2515, 2010.
- M. Harder, C. Salge, and D. Polani, “Bivariate measure of redundant information,” Physical Review E, vol. 87, no. 1, 2013.
- N. Bertschinger, J. Rauh, E. Olbrich, J. Jost, and N. Ay, “Quantifying unique information,” Entropy, vol. 16, p. 2161–2183, 2014.
- R. A. A. Ince, “Measuring multivariate redundant information with pointwise common change in surprisal,” Entropy, vol. 19, no. 7, 2017.
- R. G. James and J. P. Crutchfield., “Multivariate dependence beyond shannon information,” Entropy, 2017.
- M. Wibral, V. P. J. Kay, J. Lizier, and W. Philips, “Partial information decomposition as a unified approach to the specification of neural goal functions,” Brain and Cognition, vol. 112, 2017.
- J. Kunert-Graf, N. Sakhanenko, and D. Galas, “Partial information decomposition and the information delta: A geometric unification disentangling non-pairwise information,” Entropy, vol. 22, no. 1333, 2020.
- R. James, J. Emenheiser, and J. Crutchfield, “Unique information and secret key agreement,” MDPI Entropy, vol. 21, no. 1, 2021.
- P. Mediano, F. Rosas, A. Luppi, R. Carhart-Harris, D. Bor, A. Seth, and A. Barrett, “Towards an extended taxonomy of information dynamics via integrated information decomposition,” arxiv.org:, 2021.
- A. B. Barrett, “Exploration of synergistic and redundant information sharing in static and dynamical gaussian systems,” Phys. Rev. E., vol. 91, no. 052802, 2015.
- T. Schreiber, “Measuring information transfer,” Phys. Rev. Lett., vol. 85, 2000.
- L. Faes, D. Marinazzo, and S. Stramaglia, “Multiscale information decomposition: Exact computation for multivariate gaussian processes,” Entropy, 2017.
- A. Luppi, P. Mediano, F. Rosas, N. Holland, T. Fryer, J. O’Brien, J. Rowe, D. Menon, D.Bor, and E. Stamatakis, “A synergistic core for human brain evolution and cognition,” Nature Neuroscience, vol. 25, 2022.
- J. Massey, “Causality, feedback and directed information,” The International Symposium on Information Theory and Its Applications, 1990.
- G. Kramer, “Capacity results for the discrete memoryless network,” IEEE Transactions on Information Theory, vol. 49, no. 1, pp. 4–21, 2003.
- M. Lindner, R. Vicente, V. Priesemann, and M. Wibral, “Trentool: a matlab open source toolbox to analyse information flow in time series data with transfer entropy,” BMC Neurosci., vol. 12, p. 119, 2011.
- M. A. Kramer, E. D. Kolaczyk, and H. E. Kirsch, “Emergent network topology at seizure onset in humans,” Epilepsy Research, vol. 79, no. 173 – 186, 2008.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.