Information-related complexity: a problem-oriented approach

Abstract: A general notion of information-related complexity applicable to both natural and man-made systems is proposed. The overall approach is to explicitly consider a rational agent performing a certain task with a quantifiable degree of success. The complexity is defined as the minimum (quasi-)quantity of information that's necessary to complete the task to the given extent -- measured by the corresponding loss. The complexity so defined is shown to generalize the existing notion of statistical complexity when the system in question can be described by a discrete-time stochastic process. The proposed definition also applies, in particular, to optimization and decision making problems under uncertainty in which case it gives the agent a useful measure of the problem's "susceptibility" to additional information and allows for an estimation of the potential value of the latter.