Leveraging Machine Learning for Accurate IoT Device Identification in Dynamic Wireless Contexts

Abstract: Identifying IoT devices is crucial for network monitoring, security enforcement, and inventory tracking. However, most existing identification methods rely on deep packet inspection, which raises privacy concerns and adds computational complexity. More importantly, existing works overlook the impact of wireless channel dynamics on the accuracy of layer-2 features, thereby limiting their effectiveness in real-world scenarios. In this work, we define and use the latency of specific probe-response packet exchanges, referred to as "device latency," as the main feature for device identification. Additionally, we reveal the critical impact of wireless channel dynamics on the accuracy of device identification based on device latency. Specifically, this work introduces "accumulation score" as a novel approach to capturing fine-grained channel dynamics and their impact on device latency when training machine learning models. We implement the proposed methods and measure the accuracy and overhead of device identification in real-world scenarios. The results confirm that by incorporating the accumulation score for balanced data collection and training machine learning algorithms, we achieve an F1 score of over 97% for device identification, even amidst wireless channel dynamics, a significant improvement over the 75% F1 score achieved by disregarding the impact of channel dynamics on data collection and device latency.

• The paper introduces a novel ML approach that uses device latency and an accumulation score to uniquely fingerprint IoT devices in variable wireless conditions.
• The paper demonstrates that incorporating the accumulation score increases identification F1 accuracy from 75% to over 97%, ensuring robust performance across diverse channel utilizations.
• The paper validates its framework using tree-based models like LightGBM, enabling real-time, privacy-preserving device tracking on edge devices without relying on intrusive packet inspection.

A Machine Learning Framework for Privacy-Preserving and Accurate IoT Device Identification in Dynamic Wireless Environments

Introduction

The proliferation of IoT deployments has radically increased the diversity and number of connected devices, incentivizing robust approaches for precise device identification. Traditional solutions built on packet inspection or static address associations (e.g., IP/MAC) suffer from privacy risks, computational overhead, and vulnerability to evasion via spoofing or address randomization. Furthermore, prevailing methods largely neglect the influence of real-world wireless channel dynamics, specifically the impact of channel utilization (CU) variability on the stability and discriminative power of observation-level features, limiting practical efficacy.

This paper, "Leveraging Machine Learning for Accurate IoT Device Identification in Dynamic Wireless Contexts" (2405.17442), proposes a refined approach centered on device latency as a privacy-preserving, protocol-layer signal for device fingerprinting. Crucially, the work introduces the "accumulation score," a novel metric encapsulating instantaneous wireless channel contention surrounding probe-response exchanges. Both features are leveraged for robust, data-efficient training of modern tree-based ML models, achieving superior device identification accuracy across a spectrum of channel conditions.

Device Latency as a Fingerprinting Feature

Device latency is defined as the interval between a probe packet's complete reception and the commencement of the corresponding response's transmission by the device. Unlike round-trip time (RTT), this interval isolates device-specific processing and channel contention factors, abstracting away irrelevant AP-side and channel delays unrelated to the device under test. Accurate measurement is enabled by using a packet sniffer to correlate probe and response pairs and compute the relevant intervals.

Figure 1: Device latency ($l$) is defined as the interval between $t_4$ and $t_6$, exclusively representing device-side processing and channel contention.

Empirical observations demonstrate that device latency distributions are device-dependent but also highly sensitive to variations in channel utilization (CU). This dual dependence complicates generalization, as naive models may learn spurious associations with environmental factors rather than inherent device characteristics.

Figure 2: The testbed includes real IoT devices, background traffic generators, and monitoring infrastructure for precise, repeatable measurements.

Accumulation Score: Modeling Instantaneous Wireless Channel Conditions

The paper identifies limitations in using conventional CU measures (e.g., sampled at 10 ms intervals by commodity hardware), which offer insufficient temporal granularity relative to probe-response exchange durations. To address this, the authors introduce the accumulation score, a bespoke metric that aggregates the durations and temporal proximities of both "predecessor" (prior to response) and "successor" (immediately following response) packets, weighted by their closeness to the response event using either a Bell or Gamma curve-centric weight function.

Figure 4: Sample probe-response exchange illustrating predecessor and successor packets; the accumulation score integrates durations, intervals, and position-weighted influence.

The accumulation score quantitatively characterizes instantaneous channel contention and its relationship with each unique probe-response instance, rather than relying on coarsely averaged CU statistics. This granularity yields a strong correlation with device latency for various probe packet types and payload sizes.

Figure 6: Visualization of distinct accumulation score distributions under diverse CU conditions and different probe/response packet classes.

A critical empirical finding is the strictly increasing relationship between accumulation score and device latency—across all device types—validating its efficacy as a contextual feature that robustifies device fingerprinting against channel dynamics.

Figure 3: Device latency distributions stratified by accumulation score bins, demonstrating monotonicity and enhanced separability as accumulation increases.

Machine Learning for Context-Aware Device Identification

The foundation established by latency and accumulation score features enabled the deployment and evaluation of multiple ML algorithms, including Decision Trees, Random Forests, LightGBM, and XGBoost. The pipeline encompasses meticulous feature extraction, stratified sampling to mitigate class imbalance, and targeted experimentation to disentangle the influence of data diversity across accumulation score/CU values.

Notably, models trained on latency/score samples from a single accumulation score (i.e., CU) range generalize poorly: F1 scores plummet to as low as 25% for out-of-range test data, regardless of the number of in-range samples provided. This substantiates the claim that prior approaches, which do not account for wireless context in training, are inherently limited in operational deployments with variable channel conditions.

Figure 5: F1 device identification accuracy when training and testing on disjoint accumulation score ranges. In-range performance is high, while out-of-range generalization collapses.

By contrast, training on diverse samples spanning the full spectrum of accumulation scores (i.e., all expected CU dynamics) enables identification models to achieve consistently high accuracy. The inclusion of the accumulation score as an explicit feature further boosts accuracy, offering robustness when some latency features (e.g., those dependent on specific transport protocols) are unavailable.

Figure 7: Device identification F1 scores for multiple feature sets, demonstrating the substantial and consistent benefit of including accumulation score features—especially with limited data.

LightGBM emerged as an optimal solution, balancing accuracy, training, and inference latency, outperforming Random Forest and Decision Tree baselines. Results indicate that device identification F1 scores consistently exceed 97% on real-world testbed data when accumulation score-based balancing and feature inclusion are enforced, a significant improvement over the 75% upper bound for models trained naïvely without contextual consideration.

Figure 11: Comparative F1 scores: Latency-only features versus latency plus accumulation score, with the latter achieving consistently higher performance and stability.

Figure 8: Algorithmic performance comparison, showing accuracy (F1) and training/inference overheads for DT, RF, LGBM, and XGBoost on embedded-class processors.

Practical Implications and Research Outlook

This research establishes that high-precision, privacy-preserving IoT device identification is achievable without DPI or reliance on potentially spoofed address-level information. Incorporation of instantaneous channel context, as discretized by accumulation score, is essential for robust generalization in realistic wireless environments. From a deployment perspective, the methods described are practical for in situ execution on commercial access points leveraging current hardware acceleration and local monitoring hooks (packet sniffers, eBPF).

Implications include:

• Scalable asset tracking and anomaly detection: Improved identification enables real-time microsegmentation and rapid isolation in response to compromise or misbehavior.
• Privacy and regulatory compliance: No user-content inspection is performed, reducing privacy exposure and facilitating compliance with evolving legal mandates.
• Automated management across heterogeneous deployments: Channel context is local and fine-grained, allowing unified models across diverse hardware and traffic conditions.

Future research directions may investigate transfer learning across sites with differing nominal channel characteristics, adversarial robustness against latency/score manipulation, and feature compression/selection for even lower resource utilization.

Conclusion

This paper provides a comprehensive protocol and empirical validation for robust IoT device identification based on device latency and the accumulation score, an instantaneous metric of wireless channel contention. The methodology is orthogonal to application-layer features, respects user privacy, and is empirically shown to deliver over 97% F1 accuracy in settings where prior art is limited to 75%. The framework supports efficient ML implementation on edge devices with modest computational capability, underlining its practicality for modern, dynamic IoT deployments.