Practical Design Space Exploration

Abstract: Multi-objective optimization is a crucial matter in computer systems design space exploration because real-world applications often rely on a trade-off between several objectives. Derivatives are usually not available or impractical to compute and the feasibility of an experiment can not always be determined in advance. These problems are particularly difficult when the feasible region is relatively small, and it may be prohibitive to even find a feasible experiment, let alone an optimal one. We introduce a new methodology and corresponding software framework, HyperMapper 2.0, which handles multi-objective optimization, unknown feasibility constraints, and categorical/ordinal variables. This new methodology also supports injection of the user prior knowledge in the search when available. All of these features are common requirements in computer systems but rarely exposed in existing design space exploration systems. The proposed methodology follows a white-box model which is simple to understand and interpret (unlike, for example, neural networks) and can be used by the user to better understand the results of the automatic search. We apply and evaluate the new methodology to the automatic static tuning of hardware accelerators within the recently introduced Spatial programming language, with minimization of design run-time and compute logic under the constraint of the design fitting in a target field-programmable gate array chip. Our results show that HyperMapper 2.0 provides better Pareto fronts compared to state-of-the-art baselines, with better or competitive hypervolume indicator and with 8x improvement in sampling budget for most of the benchmarks explored.

• The paper introduces HyperMapper 2.0, a framework for multi-objective optimization in hardware design using active learning and random forests.
• It details a methodology that incorporates a beta distribution to model prior knowledge and a feasibility classifier to focus on high-potential design regions.
• Experimental results show that HyperMapper 2.0 closely approximates the Pareto front with fewer evaluations compared to traditional exhaustive search methods.

Practical Design Space Exploration

Introduction to HyperMapper 2.0

Multi-objective optimization is a pivotal aspect of computer systems design space exploration, particularly when derivative-based solutions are unavailable or impractical. HyperMapper 2.0 is a novel software framework designed to address multi-objective optimization challenges, cater to unknown feasibility constraints, and manage categorical/ordinal variables, while incorporating user prior knowledge to inform the search process. Its white-box model approach enhances interpretability, making it an adaptable tool for hardware accelerator design, particularly within the framework of the Spatial programming language.

Methodology of HyperMapper 2.0

HyperMapper 2.0 differentiates itself through its robust methodological framework that integrates several advanced techniques for optimization. The framework employs random forest-based models, recognized for capturing the discontinuities and complex behaviors typical in architectural workloads. A key innovation is the incorporation of a Beta distribution for modeling prior knowledge of design parameter distributions, facilitating targeted exploration of design spaces. This approach is augmented by an active learning strategy, iteratively refining models by observing samples around the predicted Pareto front to efficiently approximate optimal configurations.

Figure 1: Beta distribution shapes in HyperMapper 2.0.

HyperMapper 2.0's active learning component uses randomized decision forests as surrogates, iterating through adaptive sampling of design parameters to inform further exploration. It exploits a classifier for unknown feasibility constraints, achieving significant efficiency over direct, exhaustive search techniques. This classifier distinguishes feasible from infeasible regions, enhancing sampling budget efficiency by prioritizing evaluations in high-potential regions.

Evaluation and Experimental Analysis

The integration of HyperMapper 2.0 within the Spatial compiler infrastructure allows for the automatic static tuning of hardware accelerators, optimizing for minimal compute logic usage and runtime. Experimental results demonstrate the efficacy of HyperMapper 2.0, achieving superior approximated Pareto fronts with significantly reduced sampling requirements compared to baseline methods. The framework consistently improves the hypervolume indicator across diverse Spatial benchmarks, showcasing its robustness in varied application scenarios.

Figure 2: RF feasibility classifier 5-fold cross-validation recall over all benchmarks.

Notably, the feasibility classifier component demonstrates a noteworthy ability to boost recall rates across multiple benchmarks, as evidenced by the 5-fold cross-validation results. This improvement underscores the classifier's role in efficiently navigating the design space.

Optimum Versus Approximated Pareto Front

For smaller benchmarks, where exhaustive search is feasible, HyperMapper 2.0 closely approximates the true Pareto front, a testament to its optimizational proficiency. The method displays a considerable reduction in required evaluations, showcasing the framework's ability to efficiently reach near-optimal solutions with a fraction of experimental trials.

Figure 3: Optimum versus approximated Pareto front for the BlackScholes, OuterProduct, and DotProduct benchmarks.

Conclusion

HyperMapper 2.0 presents a comprehensive framework for design space exploration, integrating prior knowledge, feasibility constraints, and active learning into a single, user-friendly system. The framework aligns well with the complexities of computer system design, delivering scalable, comprehensible, and efficient optimization solutions. As future work, extending HyperMapper 2.0 to leverage full Bayesian approaches and other sampling techniques presents an exciting avenue to further enhance its capabilities, potentially broadening its applicability to other domains within system design and optimization.