Papers
Topics
Authors
Recent
Search
2000 character limit reached

Multi-objective Optimization in CPU Design Space Exploration: Attention is All You Need

Published 24 Oct 2024 in cs.LG, cs.AR, and cs.AI | (2410.18368v1)

Abstract: Design space exploration (DSE) enables architects to systematically evaluate various design options, guiding decisions on the most suitable configurations to meet specific objectives such as optimizing performance, power, and area. However, the growing complexity of modern CPUs has dramatically increased the number of micro-architectural parameters and expanded the overall design space, making DSE more challenging and time-consuming. Existing DSE frameworks struggle in large-scale design spaces due to inaccurate models and limited insights into parameter impact, hindering efficient identification of optimal micro-architectures within tight timeframes. In this work, we introduce AttentionDSE. Its key idea is to use the attention mechanism to establish a direct mapping of micro-architectural parameters to their contributions to predicted performance. This approach enhances both the prediction accuracy and interpretability of the performance model. Furthermore, the weights are dynamically adjusted, enabling the model to respond to design changes and effectively pinpoint the key micro-architectural parameters/components responsible for performance bottlenecks. Thus, AttentionDSE accurately, purposefully, and rapidly discovers optimal designs. Experiments on SPEC 2017 demonstrate that AttentionDSE significantly reduces exploration time by over 80\% and achieves 3.9\% improvement in Pareto Hypervolume compared to state-of-the-art DSE frameworks while maintaining superior prediction accuracy and efficiency with an increasing number of parameters.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Energy-performance tradeoffs in processor architecture and circuit design: a marginal cost analysis. ACM SIGARCH Computer Architecture News, 38(3):26–36, 2010.
  2. Archexplorer: Microarchitecture exploration via bottleneck analysis. In Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture, pages 268–282, 2023.
  3. Boom-explorer: Risc-v boom microarchitecture design space exploration framework. In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD), pages 1–9. IEEE, 2021.
  4. The gem5 simulator. SIGARCH Comput. Archit. News, 39(2):1–7, aug 2011.
  5. Regression transformer enables concurrent sequence regression and generation for molecular language modelling. Nature Machine Intelligence, 5(4):432–444, 2023.
  6. Leo Breiman. Random forests. Machine learning, 45:5–32, 2001.
  7. New methodology for early-stage, microarchitecture-level power-performance analysis of microprocessors. IBM Journal of Research and Development, 47(5.6):653–670, 2003.
  8. Projection pursuit gaussian process regression. IISE Transactions, 55(9):901–911, 2023.
  9. Archranker: A ranking approach to design space exploration. ACM SIGARCH Computer Architecture News, 42(3):85–96, 2014.
  10. Explainable-dse: An agile and explainable exploration of efficient hw/sw codesigns of deep learning accelerators using bottleneck analysis. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 4, ASPLOS ’23, page 87–107, New York, NY, USA, 2024. Association for Computing Machinery.
  11. Learning pareto-frontier resource management policies for heterogeneous socs: An information-theoretic approach. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pages 607–612. IEEE, 2021.
  12. A survey on ensemble learning. Frontiers of Computer Science, 14:241–258, 2020.
  13. David L Donoho. High-dimensional data analysis: The curses and blessings of dimensionality. AMS math challenges lecture, 1(2000):32, 2000.
  14. Focusing processor policies via critical-path prediction. In Proceedings of the 28th annual international symposium on Computer architecture, pages 74–85, 2001.
  15. Peter I Frazier. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811, 2018.
  16. Scar: Spatial-/channel-wise attention regression networks for crowd counting. Neurocomputing, 363:1–8, 2019.
  17. Simpoint 3.0: Faster and more flexible program phase analysis. Journal of Instruction Level Parallelism, 7(4):1–28, 2005.
  18. Evaluating associativity in cpu caches. IEEE Transactions on Computers, 38(12):1612–1630, 1989.
  19. The optimal logic depth per pipeline stage is 6 to 8 fo4 inverter delays. ACM SIGARCH Computer Architecture News, 30(2):14–24, 2002.
  20. Automated design of application specific superscalar processors: an analytical approach. In Proceedings of the 34th annual international symposium on computer architecture, pages 402–411, 2007.
  21. Illustrative design space studies with microarchitectural regression models. In 2007 IEEE 13th International Symposium on High Performance Computer Architecture, pages 340–351. IEEE, 2007.
  22. Efficient design space exploration via statistical sampling and adaboost learning. In Proceedings of the 53rd Annual Design Automation Conference, pages 1–6, 2016.
  23. Mcpat: An integrated power, area, and timing modeling framework for multicore and manycore architectures. In Proceedings of the 42nd annual ieee/acm international symposium on microarchitecture, pages 469–480, 2009.
  24. A correlation-based design space exploration methodology for multi-processor systems-on-chip. In Proceedings of the 47th Design Automation Conference, DAC ’10, page 120–125, New York, NY, USA, 2010. Association for Computing Machinery.
  25. Oscar: An optimization methodology exploiting spatial correlation in multicore design spaces. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 31(5):740–753, 2012.
  26. Environment for powerpc microarchitecture exploration. IEEE Micro, 19(3):15–25, 1999.
  27. Pioneering chiplet technology and design for the amd epyc™ and ryzen™ processor families: Industrial product. In 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), pages 57–70. IEEE, 2021.
  28. Criticality-driven superscalar design space exploration. In 2010 19th International Conference on Parallel Architectures and Compilation Techniques (PACT), pages 261–272, 2010.
  29. Branch target buffer design and optimization. IEEE transactions on computers, 42(4):396–412, 1993.
  30. Gradient boosted regression trees in scikit-learn. In PyData 2014, 2014.
  31. Mlcad: A survey of research in machine learning for cad keynote paper. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 41(10):3162–3181, 2021.
  32. The design and analysis of computer experiments, volume 1. Springer, 2003.
  33. Robert E Schapire. Explaining adaboost. In Empirical inference: festschrift in honor of vladimir N. Vapnik, pages 37–52. Springer, 2013.
  34. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1):148–175, 2015.
  35. Toward a general framework for jointly processor-workload empirical modeling. The Journal of Supercomputing, 77:5319–5353, 2021.
  36. Adaboost. rt: a boosting algorithm for regression problems. In 2004 IEEE international joint conference on neural networks (IEEE Cat. No. 04CH37541), volume 2, pages 1163–1168. IEEE, 2004.
  37. Standard Performance Evaluation Corporation. Spec cpu 2017. https://www.spec.org/cpu2017/, 2022.
  38. Moguls: a model to explore the memory hierarchy for bandwidth improvements. Acm Sigarch Computer Architecture News, 39(3):377–388, 2011.
  39. K Vanaja and RH Shobha Rani. Design of experiments: concept and applications of plackett burman design. Clinical research and regulatory affairs, 24(1):1–23, 2007.
  40. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  41. A transfer learning framework for high-accurate cross-workload design space exploration of cpu. In 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD), pages 1–9, 2023.
  42. Modse: A high-accurate multi-objective design space exploration framework for cpu microarchitectures. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2023.
  43. A high-accurate multi-objective ensemble exploration framework for design space of cpu microarchitecture. In Proceedings of the Great Lakes Symposium on VLSI 2023, GLSVLSI ’23, page 379–383, New York, NY, USA, 2023. Association for Computing Machinery.
  44. An expected hypervolume improvement algorithm for architectural exploration of embedded processors. In Proceedings of the 53rd Annual Design Automation Conference, pages 1–6, 2016.
  45. Gaussian processes for regression. Advances in neural information processing systems, 8, 1995.
  46. Active learning via transductive experimental design. In Proceedings of the 23rd international conference on Machine learning, pages 1081–1088, 2006.
  47. Expected improvement for expensive optimization: a review. Journal of Global Optimization, 78(3):507–544, 2020.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free