Interactive Preference Learning of Utility Functions for Multi-Objective Optimization

Abstract: Real-world engineering systems are typically compared and contrasted using multiple metrics. For practical machine learning systems, performance tuning is often more nuanced than minimizing a single expected loss objective, and it may be more realistically discussed as a multi-objective optimization problem. We propose a novel generative model for scalar-valued utility functions to capture human preferences in a multi-objective optimization setting. We also outline an interactive active learning system that sequentially refines the understanding of stakeholders ideal utility functions using binary preference queries.