Entropy-Based Expertise Estimation

• Entropy-based expertise estimation integrates textual, profile, and citation signals to rank academic experts using unsupervised, probabilistic evidence weighting.
• The methodology quantifies uncertainty via Shannon's entropy and fuses heterogeneous sensor outputs with Dempster–Shafer theory for robust expert ranking.
• Empirical results on large academic datasets demonstrate its competitive performance compared to supervised learning-to-rank methods and traditional aggregation models.

Entropy-based expertise estimation is a principled approach for ranking individuals' expertise in response to a query, utilizing multisensor data fusion, Shannon's entropy, and the Dempster–Shafer theory of evidence. The methodology orchestrates heterogeneous signals—textual content analysis, profile metrics, and citation graph structure—without needing supervised training, resolving sensor disagreement via probabilistic uncertainty weighting. Empirical evaluation demonstrates its efficacy in ranking academic experts, with performance matching supervised learning-to-rank methods and exceeding traditional rank aggregation baselines (Moreira et al., 2013).

1. Multisensor Framework for Expertise Estimation

The framework formalizes three independent expertise estimators ("sensors") extracting diverse forms of evidence:

• Text Sensor: Assesses information retrieval (IR)-style relevance between candidate publications and the query. Extracted features include term frequency, inverse document frequency, BM25, Jaccard similarity, Okapi-BM25 over venues, among others.
• Profile Sensor: Quantifies the candidate's productivity and publication record. Events captured include total publications, journal counts, publication years (overall and query-specific), average publications per year, etc.
• Citation Sensor: Utilizes the citation graph to measure scientific impact. Features include total citations (overall and query-specific), average citations per year, h-index variants (h, g, a, e, contemporary, trend, individual), PageRank scores of candidate’s papers, and number of unique collaborators.

For each sensor $S$, scores for a candidate $c$ are computed across all event types $e \in E_S$, normalized to $[0,1]$ via min-max normalization. A data-fusion algorithm (e.g., CombSUM) aggregates these per-sensor evidence:

$$\text{CombSUM}_S(c, q) = \sum_{e \in E_S} \text{score}_{e,\text{norm}}(c, q)$$

2. Shannon's Entropy Quantification of Sensor Uncertainty

Each sensor’s uncertainty in distinguishing expertise is quantified through Shannon’s entropy. For sensor $S$, candidate set $A$, and event set $E_S$, define:

• $relevantEvent(e, a) = 1$ if $\text{score}_e(a) > 0$, else $0$
• $$p(e,a) = \frac{relevantEvent(e,a)}{|A| \cdot |E_S|}$$

The entropy:

$$H(S) = -\sum_{a \in A} \sum_{e \in E_S} p(e,a) \cdot \log_2 p(e,a)$$

Maximum possible entropy is $\max H(S) = \log_2(|A| \cdot |E_S|)$. The normalized uncertainty weight for each sensor is:

$w_S = H(S) / \max H(S) \in [0,1]$

This weight reflects the sensor’s evidential ambiguity for candidate expertise assignments.

3. Dempster–Shafer Theory for Sensor Evidence Combination

Expertise evidences from different sensors may conflict in their candidate rank assignments. The combination protocol leverages the Dempster–Shafer framework:

• Define the frame of discernment $\Theta = \{$all candidates$\}$.
• For each sensor $S$:
  • Assign mass function $m_S$ over $2^\Theta$:
  • For singleton $\{c\}$, $m_S(\{c\}) = \text{fusion}_{S,\text{norm}}(c)$
  • For ignorance $\Theta$, $m_S(\Theta) = w_S$
  • Otherwise, $m_S(X) = 0$

Normalization ensures $\sum_{c} m_S(\{c\}) + m_S(\Theta) = 1$.

To combine two mass functions $m_1, m_2$:

• $K = \sum_{B \cap C = \emptyset} m_1(B)\,m_2(C)$
• $$m_1 \oplus m_2(A) = \frac{1}{1-K} \sum_{B \cap C = A} m_1(B)\,m_2(C)$$ for $A \neq \emptyset$; $m_1 \oplus m_2(\emptyset)=0$

Shannon's entropy modulates the mass assigned to uncertainty ($m_S(\Theta) = w_S$), integrating each sensor’s confidence into the fusion.

4. Algorithmic Realization and Final Expertise Ranking

The procedure for query $q$ consists of:

1. Candidate Retrieval: Assemble $A$, all authors with at least one document matching $q$-terms.
2. Sensor Processing:
   • Compute raw event scores $\text{score}_e(c)$ for all $e \in E_S$, $c \in A$.
   • Normalize each event’s scores to $[0,1]$ by min–max.
   • Aggregate into CombSUM per candidate.
   • Normalize CombSUM scores: $\sum_c CombSUM_S(c) = 1-w_S$.
   • Compute $H(S), \max H(S), w_S$.
   • Assign $m_S(\{c\}) = CombSUM_S(c)$, $m_S(\Theta) = w_S$.
3. Sensor Fusion:
   • Combine $$m_{\text{Text}}, m_{\text{Profile}}, m_{\text{Citation}}$$ via Dempster’s rule ($$m_{TP} = m_{\text{Text}} \oplus m_{\text{Profile}}; m_{\text{final}} = m_{TP} \oplus m_{\text{Citation}}$$).
4. Ranking: Final expertise score for candidate $c$ is $m_{\text{final}}(\{c\})$. Rank candidates in descending order.

This closed-form yields $$m_{\text{comb}}(\{c\}) = (m_1 \oplus \cdots \oplus m_n)(\{c\})$$, computed through successive Dempster–Shafer combinations.

5. Experimental Evaluation and Comparative Performance

Empirical assessment used two datasets:

• Proximity DBLP: 456,704 authors, 743,349 publications, 112,303 citations, no abstracts.
• Enriched DBLP (ArnetMiner): 1,033,050 authors, 1,632,440 publications, 2,327,450 citations, 653,514 abstracts.

Test queries comprised 13 Computer Science topics, with candidate pools of 400 authors per query and expert relevance judgments. Main metrics included Precision@$k$ ($k$=5,10,15,20) and Mean Average Precision (MAP), with statistical significance evaluated via two-sided randomization test ($\alpha=0.10$).

Key results:

Dataset	Best Fusion	P@5	MAP	Baseline MAP	Supervised MAP
Proximity DBLP	D-S+Condorcet	0.7538	0.4905	CombSUM 0.3027	—
	Text+Citation	—	0.5443
Enriched DBLP	D-S+Condorcet	0.6308	0.4055	Condorcet 0.2773	SVMmap 0.4068
	Text+Profile	—	0.4530	Model 1 0.2715	SVMrank 0.4289

CombSUM, Condorcet, and Balog’s expert finding baselines were outperformed by the proposed method, which matched the effectiveness of supervised SVMmap and SVMrank algorithms. Notably, Dempster–Shafer + entropy fusion did not require labeled training data.

6. Context, Implications, and Capabilities

The entropy-based multisensor expert estimation framework demonstrates that incorporating both uncertainty quantification and principled evidence fusion robustly resolves conflicting signals from heterogeneous academic data sources (Moreira et al., 2013). The assignment of entropy-weighted ignorance mass admits the intrinsic limitations or disagreement of each sensor into the final ranking, increasing reliability.

A plausible implication is that the method offers resilience against overfitting or adversarial profile skew, given its unsupervised nature and explicit accounting for uncertainty. Furthermore, the ability to aggregate diverse indicators (document relevance, citation influence, career productivity) in a mathematically coherent manner suggests generalizability to broader expert-finding contexts.

7. Related Methodologies and Distinctions

Compared to standard rank aggregation (CombSUM, Condorcet) and candidate/document-based probabilistic models (Balog et al.), entropy-based multisensor fusion uniquely incorporates sensor-level uncertainty via Shannon’s entropy and leverages Dempster–Shafer evidence theory for combination. Supervised learning-to-rank approaches (SVMmap, SVMrank) require labeled data, whereas the described methodology achieves comparable performance absent explicit relevance supervision.

This suggests that entropy-based expertise estimation can serve either as a standalone ranking mechanism where training data are limited, or as a complementary signal within ensemble expert finding systems.