Sequential Preference Elicitation

• Sequential preference elicitation is a computational framework that incrementally reveals latent user preferences through targeted queries and model updates.
• It employs various algorithms and query strategies, such as worst-case querying in fuzzy CSPs and Bayesian optimization, to efficiently balance query load and decision accuracy.
• Practical implementations span interactive recommendation, group decision-making, and automated planning, demonstrating reduced user burden and high query efficiency.

Sequential preference elicitation refers to a family of computational frameworks and algorithms wherein user or agent preferences—typically incomplete or latent—are incrementally uncovered through a cycle of targeted queries and model refinement. The goal is to identify optimal or near-optimal solutions with minimal user burden, balancing the number and difficulty of queries, cognitive workflow, and statistical efficiency. This paradigm is fundamental in interactive optimization, group decision-making, configurable product recommendation, social choice, and automated planning, especially where preference models are high-dimensional, personalized, or partially specified.

1. Formal Problem Definitions

Sequential preference elicitation problems are formalized in diverse mathematical and algorithmic models, including incomplete fuzzy constraint satisfaction problems (IFCSPs), random utility models, preference-based Bayesian optimization, and combinatorial social choice. A typical formalization involves:

• A search or decision space $\mathcal{X}$, often combinatorial or continuous (e.g., assignments in CSPs, routes, designs, rankings).
• A latent utility or preference function $u^*(x)$, which is unknown or only partially observed.
• A query model, specifying what can be asked (e.g., full rankings, pairwise comparisons, local preference scores, feature critiques, or sub-objective evaluations).
• A mechanism for partial information, such as missing entries in fuzzy constraints (0909.4446), incomplete rankings, or uncertainty sets over utility parameters.

A canonical case is the IFCSP $(V, D, C)$ where $V$ is a variable set, $D$ is a finite domain, and each constraint $c$ defines a possibly incomplete preference function $\mathrm{idef}: D \times D \to [0,1] \cup \{?\}$. The set $IT(P)$ consists of all missing (“$?$”) entries, and the completion of $P$ resolves all these to real values.

Relevant solution concepts include:

• Possibly optimal solution: optimal in at least one completion.
• Necessarily optimal solution: optimal in all completions.

Other domains instantiate the preference function differently, but the general structure—unknown preferences over a search space, acquired via sequential information elicitation—remains the unifying theme (0909.4446, Soufiani et al., 2013, Teso et al., 2016, Dewancker et al., 2018).

2. Core Algorithms and Frameworks

Elicitation algorithms interleave optimization or inference steps with targeted, adaptive queries. The design space is characterized by:

• Elicitation timing (“when”): Query after each search (tree-level), per branch (full assignment), or at every node (partial assignment).
• Query content (“what”): Ask for all missing preferences in a partial solution, or just the worst (minimum) partial preference, especially in fuzzy CSPs where the minimum governs the total scoring (0909.4446).
• Variable assignment order (“who”): Algorithmic ordering (based on surrogate completions), or user-guided ordering.
• Surrogate completions: Heuristic filling of uncertainties (e.g., fill all missing prefs with 0 or 1 for pessimistic/optimistic bounding).

These dimensions enable a combinatorial framework; for instance, in incomplete fuzzy CSPs, the 3-parameter design yields 16 algorithmic variants, each corresponding to a specific instantiation of (who, what, when), with a baseline random elicitation for comparison (0909.4446).

General sequential frameworks in other domains include:

• Coactive Critiquing: Iterative elicitation by improvement, supplemented with adaptive queries for feature critiques and dynamic feature space extension (Teso et al., 2016).
• Sequential Bayesian Optimization for Preferences: Models the unknown function as a GP, with human feedback as pairwise or tie-inclusive preference comparisons, and selects queries by maximizing expected improvement or information gain (Dewancker et al., 2018).
• Active Preference Search by Information Criteria: Bayesian optimal experimental design, minimizing posterior uncertainty or maximizing expected information gain about solution ranking or utility model parameters (Zhao et al., 2018, Soufiani et al., 2013).

A common meta-algorithmic structure is as follows:

1. Proposal: Compute a candidate solution using current information (e.g., by branch-and-bound, expected improvement, or greedy selection).
2. Query: Elicit missing or uncertain preference data most relevant for disambiguating optimality.
3. Update: Refine the preference model with new information, update solution set, and repeat until a termination / certainty criterion is met.

3. Query Efficiency and User Effort Metrics

Query complexity and user cognitive effort are two central metrics in sequential preference elicitation:

• Number of Elicited Preferences: Expressed as a fraction of the total missing entries $M=|IT(P)|$, i.e., $e_r=Q/M$, where $Q$ is the total number of elicitation queries (0909.4446).
• User Effort: The total number of possibilities a user must consider, especially salient when queries are complex (e.g., “what is the worst among $k$ missing preferences?” may require inspecting all $k$ tuples, even if only one value is returned) (0909.4446).
• Runtime and Anytime Quality Profiles: Solution quality and bound tightness as a function of interaction steps (often measured in wall-clock time sans user pause).

Empirical results demonstrate that the best algorithms, such as the DPI.WORST.BRANCH or SU.WORST.BRANCH variants in incomplete fuzzy CSPs, elicit only a very small fraction of missing preferences (typically $5\text{--}10\%$), with user effort similarly capped at $60\%$ or lower, even as incompleteness rises toward $95\%$ (0909.4446). Random or untargeted querying can require elicitation of nearly all missing entries.

4. Termination, Optimality, and Complexity

A key challenge is certifying solution optimality with minimal queries:

• Constraint Propagation via Surrogate Completions: In IFCSPs, constructing $P_0$ and $P_1$ (all ‘?’ as 0 or 1) enables bounding and, when both yield equal maxima, immediate termination with a provably necessarily optimal solution (0909.4446).
• Correctness and Termination Guarantees: For all 16 algorithms in (0909.4446), for any nontrivial IFCSP, the procedure is guaranteed to terminate after at most $|IT(P)|$ iterations, each strictly reducing uncertainty.
• Complexity: The worst-case query complexity is $O(|IT(P)|)$, but practical performance is much better due to query focusing.
• Fine- vs. Coarse-Grained Elicitation: In social choice, whether preferences are elicited as full rankings or as pairwise comparisons (find vs coarse mode) can radically affect both the computational difficulty of termination checks and the overall number of required queries (0903.1137).

5. Experimental Findings and Trade-offs

Comprehensive empirical studies reveal:

• SU.WORST.BRANCH consistently minimizes queries and user effort if users can provide ordering guidance; DPI.WORST.BRANCH is robust when user cooperation is limited (0909.4446).
• Query timing (elicit-at-branch vs elicit-at-node vs after full search) notably influences efficiency: branch-level is always superior unless branching is extremely frequent.
• For binary (hard) CSPs, phase transitions affect elicitation: even at maximal complexity, DPI.WORST.BRANCH elicits only $\sim20\%$ of missing prefs.
• The same pattern holds across more complex or structured problem classes, such as incomplete temporal CSPs.

Summary of algorithmic trade-offs:

Variant	Elicitation Ratio	User Effort	User Help Needed
SU.WORST.BRANCH	$<$5%	$<$60%	Yes (“smart” user)
DPI.WORST.BRANCH	$<$10%	$<$60%	No
Random	$\approx$100%	$\approx$100%	No

6. Practical Guidelines and Theoretical Directions

Empirical and theoretical analysis yield actionable recommendations:

• Preferred strategies: Use SU.WORST.BRANCH whenever user assistance in ordering is feasible; otherwise, default to DPI.WORST.BRANCH for minimal query/effort overhead (0909.4446).
• Query design: Use “worst” when possible: asking for the worst among missing preferences in a solution suffices for fuzzy CSP optimality and dramatically reduces total queries.
• When user cost is limiting: Substitute more queries of simpler form (e.g., What=all) for fewer but higher-cognitive-load queries if user's time per evaluation is expensive.
• Robustness to incomplete or interval-bounded preferences: Open questions remain regarding partial interval or set-valued preference information and their integration into the minimal elicitation framework.
• Algorithmic extensions: Integration of elicitation cost into variable and value ordering heuristics, local-search algorithms that query just-in-time, and handling of weighted or non-idempotent semiring constraints are active research frontiers (0909.4446).
• Complexity-theoretic boundaries: For voting and social choice, the choice of coarse vs fine elicitation, the voting rule, and preference domain restrictions (e.g., single-peakedness) directly affect both query and computational complexity of deciding when sufficient information has been obtained (0903.1137).

7. Impact and Extensions

Sequential preference elicitation methodologies are applicable beyond fuzzy CSPs, encompassing interactive recommendation, constructive configuration, active ranking, group decision, and social choice. The paradigm scales to:

• Feature and objective discovery: Dynamically learn not just preference values but new criteria or sub-objectives from user input, as in coactive critiquing (Teso et al., 2016).
• Bayesian and information-theoretic active querying: Apply acquisition functions (expected improvement, mutual information, minimal expected regret) to optimize query selection (Dewancker et al., 2018, Soufiani et al., 2013, Zhao et al., 2018).
• Robust and uncertainty-aware decision-making: Explicitly handle set-based or distributional uncertainty over preferences, optimizing for worst-case utility or regret within sequential query loops (Vayanos et al., 2020).
• Combinatorial and structured domains: Leverage structural properties (single-peakedness on graphs, treewidth, single-crossing) to provably minimize query complexity (Dey et al., 2016, Dey et al., 2016).

In summary, sequential preference elicitation offers a powerful, theoretically sound set of tools and algorithmic design patterns for uncovering optimal or near-optimal solutions under preference uncertainty, with demonstrated empirical efficiency and rigorous complexity analysis across soft, hard, and structured constraint domains (0909.4446).