2000 character limit reached
TransOpt: Transformer-based Representation Learning for Optimization Problem Classification
Published 29 Nov 2023 in cs.LG and math.OC | (2311.18035v1)
Abstract: We propose a representation of optimization problem instances using a transformer-based neural network architecture trained for the task of problem classification of the 24 problem classes from the Black-box Optimization Benchmarking (BBOB) benchmark. We show that transformer-based methods can be trained to recognize problem classes with accuracies in the range of 70\%-80\% for different problem dimensions, suggesting the possible application of transformer architectures in acquiring representations for black-box optimization problems.
- U. Škvorc, T. Eftimov, and P. Korošec, “Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis,” Applied Soft Computing, vol. 90, p. 106138, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1568494620300788
- G. Cenikj, R. Dieter Lang, A. Petrus Engelbrecht, C. Doerr, P. Korošec, and T. Eftimov, “SELECTOR: Selecting a Representative Benchmark Suite for Reproducible Statistical Comparison,” in Proceedings of The Genetic and Evolutionary Computation Conference, 2022, in Press.
- M. A. Muñoz and K. Smith-Miles, “Generating new space-filling test instances for continuous black-box optimization,” Evolutionary Computation, vol. 28, pp. 379–404, 09 2020.
- K. M. Malan and A. P. Engelbrecht, “A survey of techniques for characterising fitness landscapes and some possible ways forward,” Information Sciences, vol. 241, pp. 148–163, 2013. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0020025513003125
- O. Mersmann, B. Bischl, H. Trautmann, M. Preuss, C. Weihs, and G. Rudolph, “Exploratory landscape analysis,” in Proceedings of the 13th annual conference on Genetic and evolutionary computation, 2011, pp. 829–836.
- G. Petelin, G. Cenikj, and T. Eftimov, “Tla: Topological landscape analysis for single-objective continuous optimization problem instances,” in 2022 IEEE Symposium Series on Computational Intelligence (SSCI), 2022, pp. 1698–1705.
- M. V. Seiler, R. P. Prager, P. Kerschke, and H. Trautmann, “A collection of deep learning-based feature-free approaches for characterizing single-objective continuous fitness landscapes,” in Proceedings of the Genetic and Evolutionary Computation Conference, ser. GECCO ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 657–665. [Online]. Available: https://doi.org/10.1145/3512290.3528834
- N. van Stein, F. X. Long, M. Frenzel, P. Krause, M. Gitterle, and T. Bäck, “Doe2vec: Deep-learning based features for exploratory landscape analysis,” 03 2023.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2017. [Online]. Available: https://arxiv.org/abs/1706.03762
- N. Hansen, S. Finck, R. Ros, and A. Auger, “Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions,” INRIA, Research Report RR-6829, 2009. [Online]. Available: https://hal.inria.fr/inria-00362633
- N. Hansen, A. Auger, R. Ros, O. Mersmann, T. Tušar, and D. Brockhoff, “Coco: A platform for comparing continuous optimizers in a black-box setting,” Optimization Methods and Software, pp. 1–31, 2020.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.