A Reliability Theory of Compromise Decisions for Large-Scale Stochastic Programs
Abstract: Stochastic programming models can lead to very large-scale optimization problems for which it may be impossible to enumerate all possible scenarios. In such cases, one adopts a sampling-based solution methodology in which case the reliability of the resulting decisions may be suspect. For such instances, it is advisable to adopt methodologies that promote variance reduction. One such approach goes under a framework known as "compromise decision", which requires multiple replications of the solution procedure. This paper studies the reliability of stochastic programming solutions resulting from the "compromise decision" process. This process is characterized by minimizing an aggregation of objective function approximations across replications, presumably conducted in parallel. We refer to the post-parallel-processing problem as the problem of "compromise decision". We quantify the reliability of compromise decisions by estimating the expectation and variance of the "pessimistic distance" of sampled instances from the set of true optimal decisions. Such pessimistic distance is defined as an estimate of the largest possible distance of the solution of the sampled instance from the "true" optimal solution set. The Rademacher average of instances is used to bound the sample complexity of the compromise decision.
- T. Asamov and W. B. Powell. Regularized decomposition of high-dimensional multistage stochastic programs with markov uncertainty. SIAM Journal on Optimization, 28(1):575–595, 2018.
- Spectrally-normalized margin bounds for neural networks. Advances in Neural Information Processing Systems, 30, 2017.
- P. L. Bartlett and S. Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3(Nov):463–482, 2002.
- D. Bertsimas and J. N. Tsitsiklis. Introduction to linear optimization, volume 6. Athena Scientific Belmont, MA, Belmont, 1997.
- Distributionally robust optimization and robust statistics. arXiv preprint arXiv:2401.14655, 2024.
- Theory of classification: A survey of some recent advances. ESAIM: probability and statistics, 9:323–375, 2005.
- Y. Carson and A. Maria. Simulation optimization: methods and applications. In Proceedings of the 29th conference on Winter simulation, pages 118–126, 1997.
- A. Deo and K. Murthy. Achieving efficiency in black-box simulation of distribution tails with self-structuring importance samplers. Operations Research, 2023.
- R. Durrett. Probability: theory and examples, volume 49. Cambridge university press, Cambridge, 2019.
- Normalized convergence in stochastic optimization. Annals of Operations Research, 30(1):187–198, 1991.
- Sample average approximation method for compound stochastic optimization problems. SIAM Journal on Optimization, 23(4):2231–2263, 2013.
- H. Gangammanavar and S. Sen. Stochastic dynamic linear programming: A sequential sampling algorithm for multistage stochastic linear programming. SIAM Journal on Optimization, 31(3):2111–2140, 2021.
- P. Glasserman and J. Li. Importance sampling for portfolio credit risk. Management science, 51(11):1643–1656, 2005.
- Regularized stochastic dual dynamic programming for convex nonlinear optimization problems. Optimization and Engineering, 21:1133–1165, 2020.
- J. L. Higle. Variance reduction and objective function evaluation in stochastic linear programs. INFORMS J. Comput., 10:236–247, 1998.
- J. L. Higle and S. Sen. Finite master programs in regularized stochastic decomposition. Mathematical Programming, 67(1):143–168, 1994.
- T. Homem-de Mello and G. Bayraksan. Stochastic constraints and variance reduction techniques. In Handbook of simulation optimization, pages 245–276. Springer, 2014.
- M. Kääriäinen and T. Elomaa. Rademacher penalization over decision tree prunings. In European Conference on Machine Learning, pages 193–204. Springer, 2003.
- R. Kawai. Optimizing adaptive importance sampling by stochastic approximation. SIAM Journal on Scientific Computing, 40(4):A2774–A2800, 2018.
- J. E. Kelley, Jr. The cutting-plane method for solving convex programs. Journal of the Society for Industrial and Applied Mathematics, 8(4):703–712, 1960.
- K. C. Kiwiel. Proximity control in bundle methods for convex nondifferentiable minimization. Mathematical Programming, 46(1-3):105–122, 1990.
- Simulation-based optimization with stochastic approximation using common random numbers. Management Science, 45(11):1570–1578, 1999.
- The sample average approximation method for stochastic discrete optimization. SIAM Journal on Optimization, 12(2):479–502, 2002.
- V. KozmÃk and D. P. Morton. Evaluating policies in risk-averse multi-stage stochastic programming. Mathematical Programming, 152:275–300, 2015.
- J. Liu and S. Sen. Asymptotic results of stochastic decomposition for two-stage stochastic quadratic programming. SIAM Journal on Optimization, 30(1):823–852, 2020.
- M. Mohri and A. Rostamizadeh. Rademacher complexity bounds for non-iid processes. Advances in Neural Information Processing Systems, 21, 2008.
- Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization, 19(4):1574–1609, 2009.
- Inexact bundle methods for two-stage stochastic programming. SIAM Journal on Optimization, 21(2):517–544, 2011.
- R. Pasupathy and Y. Song. Adaptive sequential sample average approximation for solving two-stage stochastic linear programs. SIAM Journal on Optimization, 31(1):1017–1048, 2021.
- Multi-stage stochastic optimization applied to energy planning. Mathematical programming, 52(1):359–375, 1991.
- A. B. Philpott and Z. Guan. On the convergence of stochastic dual dynamic programming and related methods. Operations Research Letters, 36(4):450–455, 2008.
- R. T. Rockafellar. Monotone operators and the proximal point algorithm. SIAM Journal on Control and Optimization, 14(5):877–898, 1976.
- A. Ruszczyński. A regularized decomposition method for minimizing a sum of polyhedral functions. Mathematical Programming, 35:309–333, 1986.
- Network planning with random demand. Telecommunication Systems, 3(1):11–30, 1994.
- S. Sen and Y. Liu. Mitigating uncertainty via compromise decisions in two-stage stochastic linear programming: Variance reduction. Operations Research, 64(6):1422–1437, 2016.
- A. Shapiro. Monte carlo sampling methods. Handbooks in Operations Research and Management Science, 10:353–425, 2003.
- A. Shapiro. On complexity of multistage stochastic programs. Operations Research Letters, 34(1):1–8, 2006.
- A. Shapiro and A. Nemirovski. On complexity of stochastic programming problems. Continuous optimization: Current trends and modern applications, pages 111–146, 2005.
- S. Sun. Multi-view laplacian support vector machines. In Advanced Data Mining and Applications: 7th International Conference, ADMA 2011, Beijing, China, December 17-19, 2011, Proceedings, Part II 7, pages 209–222. Springer, 2011.
- R. M. Van Slyke and R. Wets. L-shaped linear programs with applications to optimal control and stochastic programming. SIAM Journal on Applied Mathematics, 17(4):638–663, 1969.
- J. Xu and S. Sen. Compromise policy for multi-stage stochastic linear programming: Variance and bias reduction. Computers & Operations Research, 153:106132, 2023.
- J. Xu and S. Sen. Ensemble variance reduction methods for stochastic mixed-integer programming and their application to the stochastic facility location problem. INFORMS Journal on Computing, 36(2):587–599, 2024.
- In-memory big data management and processing: A survey. IEEE Transactions on Knowledge and Data Engineering, 27(7):1920–1948, 2015.
- P. Zhao and T. Zhang. Stochastic optimization with importance sampling for regularized loss minimization. In International Conference on Machine Learning, pages 1–9. PMLR, 2015.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.