D-KalmanNet Architecture for Adaptive Filtering

• D-KalmanNet is a hybrid filtering architecture that replaces analytic Kalman gain with a learnable RNN module for adaptive state estimation in linear and mildly nonlinear systems.
• The system leverages diverse training protocols—supervised, unsupervised, federated, and adaptive—to robustly manage nonstationary or unknown noise statistics.
• Advanced variants such as FedKalmanNet and Bayesian KalmanNet extend its capabilities with privacy-preserving collaboration, uncertainty quantification, and rapid parameter adaptation.

The D-KalmanNet architecture denotes a class of data-driven, recurrent neural network (RNN)–augmented Kalman filters, in which standard filter elements—particularly the Kalman gain and, thus, error covariance computation—are replaced by neural network modules. D-KalmanNet and its derivatives are designed for state estimation in linear or mildly nonlinear dynamical systems where latent process and measurement noise statistics are unknown or nonstationary. The architecture is characterized by its hybridization of interpretable Kalman filter structure with learned, parametric gain computation and covariate tracking, leveraging supervised, unsupervised, federated, and adaptive (hypernetwork-) training protocols. It enables reliable filtering, efficient adaptation, and, in specialized variants, uncertainty quantification and federated collaboration across distributed agents.

1. Theoretical Model Foundations

D-KalmanNet operates within the discrete-time linear state-space model: $$\mathbf{x}_t = F\,\mathbf{x}_{t-1} + \mathbf{e}_t, \quad \mathbf{e}_t \sim \mathcal{N}(0, Q_t)$$

$$\mathbf{y}_t = H\,\mathbf{x}_t + \mathbf{v}_t, \quad \mathbf{v}_t \sim \mathcal{N}(0, R_t)$$

where the transition and observation operators $F$ and $H$ are known, while $Q_t$ and $R_t$ can be unknown and time-varying (Ni et al., 2023, Revach et al., 2021, Dahan et al., 2023).

Classical Kalman Filtering computes a recursive estimate $\hat{\mathbf{x}}_{t|t}$ of the latent state using model-based prediction and an analytic Kalman gain: $$\hat{\mathbf{x}}_{t|t} = \hat{\mathbf{x}}_{t|t-1} + K_t\Bigl(\mathbf{y}_t - \hat{\mathbf{y}}_{t|t-1}\Bigr)$$ This gain $K_t$ is precisely optimal under Gaussian noise and known $Q_t$, $R_t$, but becomes suboptimal or inapplicable when these parameters are unknown.

2. Core D-KalmanNet Architecture

D-KalmanNet replaces analytic gain computation by a parametric RNN that maps observed innovations to an estimate of the Kalman gain at each time step. Structurally, the architecture can be segmented into:

• Model-based prediction module: Updates the predicted state $$\hat{\mathbf{x}}_{t|t-1} = F\hat{\mathbf{x}}_{t-1|t-1}$$ and predicted observation $$\hat{\mathbf{y}}_{t|t-1} = H\hat{\mathbf{x}}_{t|t-1}$$ (without learned parameters).
• Innovation computation: Calculates $$\Delta\mathbf{y}_t = \mathbf{y}_t - \hat{\mathbf{y}}_{t|t-1}$$.
• Gain Estimator (RNN subnetwork): Typically a GRU or LSTM cell processes a feature vector (e.g., innovation, predicted state, velocity inputs) to produce the vectorized Kalman gain $K_t$. This block can be implemented in multiple layers: feature extraction (e.g., fully connected + ReLU), recurrent update (e.g., GRU), and final mapping (fully connected linear) (Revach et al., 2021, Ni et al., 2023).
• Update block: Uses the learned gain to correct the predicted state via the standard Kalman update rule.

A generic configuration is summarized in the table:

Module	Input	Key Parameters
Model-based prediction	$\hat{\mathbf{x}}_{t-1|t-1}$	$F$, $H$
Gain estimator (RNN)	Innovations, prior updates	weights $\theta$
RNN cell	GRU/LSTM, tanh gates	hidden size $d_h$
Output (gain)	RNN hidden state	FC, weights

3. Training Methodologies

D-KalmanNet can be trained in supervised, unsupervised, federated, or adaptive paradigms:

• Supervised learning: Minimizes the empirical $\ell_2$ loss between estimated and true states over datasets of known state-observation trajectories (Ni et al., 2023, Dahan et al., 2023).
• Unsupervised training: Utilizes the predicted next-observation as a self-supervision target, computing loss as mean squared error between $\mathbf{y}_t$ and $\hat{\mathbf{y}}_{t|t-1}$ (which depends on previous update steps) (Revach et al., 2021).
• Federated learning: Multiple clients (e.g., vehicles) train local D-KalmanNet models on private trajectory data, exchanging only model weights with a central server for aggregation (FedAvg) and redistribution. This protocol enables collaborative improvement of a global filter without raw data sharing. Steps include local adaptation (SGD training), weight upload, global averaging, and broadcast of updated global weights (Piperigkos et al., 2024).
• Adaptive context-modulation (AKNet): A hypernetwork processes a context descriptor (e.g., “Scale-of-W”) and produces neuron-wise gain and shift adapters for the main RNN, enabling rapid parameter adaptation to noise statistics changes without retraining the entire network (Ni et al., 2023).

4. Advanced Variants and Extensions

Several D-KalmanNet variants augment the basic framework:

• FedKalmanNet: Integrates federated learning for distributed, privacy-preserving training across a fleet of agents. Each client trains locally with its own data, periodically synchronizing with a shared global model by averaging weights. Communication requires only model parameters ($\sim$20K per round), with no raw sensor data exchanged (Piperigkos et al., 2024).
• Bayesian KalmanNet: Implements Monte Carlo dropout (concrete dropout) at training and inference to estimate predictive uncertainty and obtain empirical error covariance. This is achieved by sampling network weight masks at each forward pass and using the variance of the prediction ensemble as an uncertainty measure, thus extracting an error-covariance estimate aligned with Kalman filter principles (Dahan et al., 2023).
• Adaptive KalmanNet (AKNet): Wraps the core gain computation DNN in a compact hypernetwork. By transforming a context scalar into layer-wise gain and shift modulations, the system dynamically tunes itself to current noise conditions without full retraining, mimicking Kalman filter adaptability but within a data-driven regime (Ni et al., 2023).

5. Algorithmic Details and Implementation

The prediction-update pipeline can be described compactly:

1. Prediction:

$$\hat{\mathbf{x}}_{t|t-1} = F\hat{\mathbf{x}}_{t-1|t-1}$$

$$\hat{\mathbf{y}}_{t|t-1} = H\hat{\mathbf{x}}_{t|t-1}$$

1. Feature formation: Concatenate relevant signals (e.g., innovation $\mathbf{y}_t - \hat{\mathbf{y}}_{t|t-1}$, prior state, control inputs).
2. Gain estimation: Feed feature vector into RNN (GRU/LSTM) to infer $K_t$ as a vector, then reshape to $m \times n$.
3. Correction:

$$\hat{\mathbf{x}}_{t|t} = \hat{\mathbf{x}}_{t|t-1} + K_t(\mathbf{y}_t - \hat{\mathbf{y}}_{t|t-1})$$

1. Loss computation:
   • Supervised: Minimize $$\frac{1}{T} \sum_t \|\hat{\mathbf{x}}_t - \mathbf{x}_t\|^2 + \gamma \|\theta\|^2$$
   • Unsupervised: Minimize $$\frac{1}{T} \sum_t \|\mathbf{y}_t - \hat{\mathbf{y}}_{t|t-1}\|^2 + \gamma \|\theta\|^2$$
   • Bayesian: Empirical mixture of prediction MSE and diagonal covariance matching, combined with per-layer KL regularization on dropout rates (Dahan et al., 2023).

6. Communication Protocols in Distributed Settings

In the federated FedKalmanNet protocol (Piperigkos et al., 2024):

• Initialization: The server initializes global weights $\boldsymbol\theta_g^{(0)}$.
• FL round:
  • The server broadcasts $\boldsymbol\theta_g^{(m-1)}$ to all clients.
  • Each client sets $$\boldsymbol\theta_i \leftarrow \boldsymbol\theta_g^{(m-1)}$$, executes $E$ epochs of local SGD on its dataset, and returns $\boldsymbol\theta_i^{(m)}$.
  • The server aggregates $$\boldsymbol\theta_g^{(m)} = \sum_i a_i\boldsymbol\theta_i^{(m)}$$ (usually $a_i = 1/N$).
• Privacy: Only model weights are transmitted; no raw data ever leave local clients.

7. Interpretability, Adaptation, and Uncertainty Quantification

• Interpretability: D-KalmanNet maintains the prediction/correction decomposition of the classical Kalman filter, providing clear correspondence between learned and analytic filter quantities (Revach et al., 2021, Dahan et al., 2023).
• Adaptation: Through online or federated updates, D-KalmanNet variants flexibly adapt to time-varying or unknown process/measurement statistics. Unsupervised loss enables adaptation to unlabelled streams, while hypernetwork-enhanced AKNet provides rapid, data-efficient parameter modulation (Ni et al., 2023).
• Uncertainty: Bayesian KalmanNet estimates empirical error covariance via Monte Carlo dropout ensembles, making it suitable for applications where uncertainty quantification is essential (Dahan et al., 2023).

D-KalmanNet and its variants demonstrate robust performance across canonical filtering tasks, resilient adaptation to nonstationary noise, privacy-preserving collaborative training, and uncertainty-aware inference, while preserving the rigorous structure of model-based filtering (Piperigkos et al., 2024, Ni et al., 2023, Dahan et al., 2023, Revach et al., 2021).