Kernel-Weighted Local Likelihood Estimators

• Kernel-weighted local likelihood estimators are nonparametric methods that use localized polynomial approximations and kernel weights to model local density behavior.
• They deliver robust results in boundary, tail, and multivariate settings by incorporating transformation techniques and adaptive bandwidth selection.
• These estimators offer simultaneous estimation of density and its derivatives while controlling bias and variance through optimized local likelihood maximization.

A kernel-weighted local likelihood estimator is a nonparametric estimation methodology where, instead of fitting a global parametric model, the local behavior of the density or parameter is modeled via polynomial expansion or local parametric approximation, with fitting performed using a kernel-weighted (localized) likelihood. This approach includes classical local-likelihood density estimation, transformation-based schemes for boundary-affected problems (notably on $\mathbb{R}_+$), multivariate density and derivative estimation, and recent developments in localized inference for regression-type or copula models.

1. General Formulation of Kernel-Weighted Local Likelihood Estimators

Let $X$ be a random variable (univariate or multivariate) with density $f_X$. The kernel-weighted local likelihood estimator constructs, around each point $x$, a localized version of the log-likelihood, replacing the population density by a local polynomial (in log-scale) or a parametric approximation. For a transformation $T:(0,\infty)\to\mathbb{R}$, $Y=T(X)$, the general kernel-weighted local log-likelihood at $x$ (equivalently at $y_0=T(x)$) is

$$L(\theta\,;\,x,h) = \sum_{i=1}^n K_h\bigl(T(X_i)-T(x)\bigr)\, \ell\bigl(\theta\,;\,T(X_i)\bigr) - n\int K_h\bigl(t-T(x)\bigr)\, f_Y\bigl(t;\theta\bigr)\,dt,$$

where $K_h(u) = h^{-1}K(u/h)$ is a kernel weight and $\ell(\theta; y) = \log f_Y(y; \theta)$ (Geenens et al., 2016). For multivariate $X\in\mathbb{R}^d$, this extends to local quadratic log-density models using a $d$-variate kernel and vectorized local moments (Strähl et al., 2018).

The parameter vector $\theta$ may be a local polynomial expansion, e.g., for degree $p$,

$$\log f_Y(y) \approx a_0 + a_1 (y-y_0) + \cdots + a_p (y-y_0)^p.$$

The local estimator $\widetilde{\boldsymbol{a}}(y_0)$ is the maximizer of the corresponding local log-likelihood.

2. Methodological Variants and Extensions

Transformation for Support Adaptation: For densities on $(0, \infty)$, common transformations include $T(x) = \log x$ and (for better exponential tail handling) the "probex" transformation $T(x) = \Phi^{-1}(1 - e^{-x})$. The estimator for $f_X$ is then obtained by back-transformation: $$\tilde{f}_X^{(T,p)}(x) = \tilde f_Y^{(p)}(T(x))\, T'(x),$$ where $\tilde f_Y^{(p)}$ is the local-likelihood density estimate of $Y$ (Geenens et al., 2016).

Multivariate Density and Derivative Estimation: For $X\in\mathbb{R}^d$, local quadratic expansions yield simultaneous estimators for the log-density, its gradient, and Hessian (second derivatives). The Gaussian kernel admits closed-form solutions for the local estimator triplet $(\hat c, \hat{\b}, \hat{\A})$ corresponding to $(\log f(x), D\log f(x), D^2\log f(x))$ (Strähl et al., 2018).

Local Likelihood in Regression-Type and Copula Models: In models where parameters (e.g., in a copula $c(u|\theta)$) vary with covariate $y$, a local-polynomial basis is used to locally approximate a transformed calibration function $v(y) = \psi(\theta(y))$, leading to the kernel-weighted local log-likelihood: $$L_n(\beta;u) = \frac{1}{n h^s} \sum_{i=1}^n K\big((Y_i-u)/h\big)\, \ell\big(\psi^{-1}(\beta^T Z_{i,u}),\,U_i\big)$$ where $Z_{i,u}$ is the local polynomial basis at $u$ and $U_i$ are pseudo-observations; the local MLE $\widehat{\beta}(u)$ targets the intercept $v(u)$ and hence $\theta(u)$ (Muia, 4 Jan 2026).

3. Asymptotic Properties and Optimal Bandwidth

Bias and Variance: For the local-likelihood transformation kernel density estimator (LLTKDE) of order $p$,

$$\sqrt{nh}\left(\tilde f_X^{(T,p)}(x) - f_X(x) - \frac12 h^2 b_T^{(p)}(x)\right) \overset{\mathcal{L}}{\longrightarrow} \mathcal{N}(0,\nu_p v_T^2(x)),$$

where $v_T^2(x) = T'(x)f_X(x)$, with explicit forms for $\nu_p$ and $b_T^{(p)}(x)$ depending on $p$ and kernel moments (Geenens et al., 2016).

Rates of Convergence: For local log-quadratic density estimation ($p=2$), the bias is $O(h^4)$, and the mean squared error (MSE) rate is $n^{-8/9}$ in the univariate case (Geenens et al., 2016). In the multivariate case, under $f\in \mathcal{C}_b^4(\mathbb{R}^d)$, the optimal honest rates for simultaneous estimation of the log-density and its derivatives are: $\E\{(\hat\ell-\ell)^2\}\asymp n^{-8/(d+8)},\quad \E\{\|\widehat{D\ell}-D\ell\|^2\}\asymp n^{-4/(d+8)}$ with bandwidth $h\asymp n^{-1/(d+8)}$ (Strähl et al., 2018).

Uniform Consistency: In covariate-dependent local-likelihood, e.g., for copula parameters,

$$\sup_{u\in U_0} \|\widehat\theta(u) - \theta(u)\| = O_p(h^{p+1} + \frac{\log(1/h)}{n h^s}),$$

with uniform asymptotic expansions governed by empirical process entropy bounds (Muia, 4 Jan 2026). The optimal uniform bandwidth rate is

$$h_{\rm opt} \asymp \biggl(\frac{\log n}{n}\biggr)^{1/(2(p+1)+s)}.$$

4. Bandwidth Selection, Kernel Choice, and Practical Implementation

Kernel Functions: Any smooth, symmetric kernel is admissible. Gaussian, Epanechnikov, and compactly supported kernels are commonly used, satisfying normalization and moment conditions (Geenens et al., 2016, Strähl et al., 2018, Muia, 4 Jan 2026).

Bandwidth Selection: Fixed bandwidth $h$ can be selected by least-squares cross-validation (LSCV) on the transformed or covariate scale, minimizing

$$\mathrm{LSCV}(h) = \int\{\tilde f_Y^{(p)}(y)\}^2\,dy - \frac{2}{n}\sum_{i=1}^n\tilde f_{Y(-i)}^{(p)}(Y_i).$$

Nearest-neighbour (NN) bandwidths, $h(y)=|y-Y_{(\lfloor n\alpha\rfloor) y}|$, chosen by cross-validation over $\alpha$, adapt locally to data sparsity, especially useful for boundary and heavy-tail stabilization (Geenens et al., 2016).

Numerical Fitting: Implementations such as the R package locfit efficiently solve the localized log-likelihood maximization and bandwidth selection for both univariate and multivariate settings (Geenens et al., 2016).

5. Comparative Performance and Use Cases

Boundary and Tail Behavior: LLTKDE outperforms classical reflection, cut-and-normalise, boundary-corrected kernel estimators, and Gamma-kernel approaches for densities supported on $\mathbb{R}_{+}$—notably near $x=0$ and in the right tail—due to reduced boundary bias ($O(h^2)$ for $p=1$, $O(h^4)$ for $p=2$) and adaptive variance properties (Geenens et al., 2016). The improvement is most significant where classical approaches fail due to lack of support adaptation or inappropriate variance scaling.

Multivariate and Log-Derivative Estimation: The local log-likelihood framework, as opposed to direct kernel differentiation, yields non-negative density estimators by construction, matches the best attainable convergence rates, and provides simultaneous consistent estimates of derivatives (Strähl et al., 2018).

Covariate-Dependent Models: In conditional copula settings, kernel-weighted local likelihood estimators facilitate nonparametric recovery of smoothly varying association structures, enabling uniform statistical guarantees necessary for simultaneous inference (such as uniform confidence bands over the covariate domain) (Muia, 4 Jan 2026).

6. Algorithmic Summary and Workflow

The kernel-weighted local likelihood estimation procedure is summarized as follows for the univariate positive-support case (Geenens et al., 2016):

1. Select transformation $T$ (log or probex, depending on prior or expected exponential near-boundary behavior).
2. Transform sample: $Y_i = T(X_i)$.
3. Fit local log-polynomial ($p=2$ recommended) density estimate $\tilde f_Y^{(2)}$ using fixed/NN bandwidth selected by cross-validation.
4. Back-transform: compute $\hat f_X(x) = \tilde f_Y^{(2)}(T(x))\,T'(x)$.
5. Diagnostics: Visual fit assessment or cross-validation diagnostics on an appropriate interval $(0, q_{0.999})$.

For multivariate or regression-type/covariate settings, the process generalizes to local polynomial approximation in the relevant variables, kernel-weighted score/hessian computation, and bandwidth selection as described above (Strähl et al., 2018, Muia, 4 Jan 2026).

7. Simulation Evidence and Real-Data Applications

Monte Carlo studies on a variety of prototypical positive densities and real data (suicide-spell durations, ozone levels, wage data) demonstrate that local-likelihood transformation kernel estimators (with log and probex transforms, $p=2$) consistently yield lower integrated absolute relative error in boundary and tail regions, with smooth estimates avoiding over-smoothing of modes or shoulders (Geenens et al., 2016). In multivariate and covariate-dependent models, the method ensures stable optimization and reliable local inference across the entire covariate domain (Strähl et al., 2018, Muia, 4 Jan 2026).