The Unfinished Search for Wave-Particle and Classical-Quantum Harmony

Abstract: The main purpose of this paper is to review the progress that has taken place so far in the search for a single unifying principle that harmonizes (i) the wave and particle natures of matter and radiation, both at the quantum and the classical levels, on the one hand and (ii) the classical and quantum theories of matter and radiation on the other hand. The famous paradoxes of quantum theory, the mysterious nature of measurements in quantum theory and the principal no-go theorems for hidden variables are first briefly reviewed. The Koopman-von Neumann Hilbert space theory based on complex wave functions underlying particle trajectories in classical phase space, is an important step forward in that direction. It provides a clear and beautiful harmony of classical waves and particles. Sudarshan has given an alternative but equivalent formulation that shows that classical mechanics can be regarded as a quantum theory with essentially hidden non-commuting variables. An extension of KvNS theory to classical electrodynamics provides a sound Hilbert space foundation to it and satisfactorily accounts for entanglement and Bell-CHSH-like violations already observed in classical polarization optics. An important new insight that has been obtained through these developments is that entanglement and Bell-like inequality violations are neither unique signatures of quantumness nor of non-locality---they are rather signatures of non-separability. Finally, Sudarshan's proposed solution to the measurement problem using KvNS theory for the measuring apparatus is sketched to show to what extent wave and particles can be harmonized in quantum theory.