Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Reinforcement Learning Approach to Sensing Design in Resource-Constrained Wireless Networked Control Systems

Published 1 Apr 2022 in eess.SY, cs.LG, cs.RO, and cs.SY | (2204.00703v5)

Abstract: In this paper, we consider a wireless network of smart sensors (agents) that monitor a dynamical process and send measurements to a base station that performs global monitoring and decision-making. Smart sensors are equipped with both sensing and computation, and can either send raw measurements or process them prior to transmission. Constrained agent resources raise a fundamental latency-accuracy trade-off. On the one hand, raw measurements are inaccurate but fast to produce. On the other hand, data processing on resource-constrained platforms generates accurate measurements at the cost of non-negligible computation latency. Further, if processed data are also compressed, latency caused by wireless communication might be higher for raw measurements. Hence, it is challenging to decide when and where sensors in the network should transmit raw measurements or leverage time-consuming local processing. To tackle this design problem, we propose a Reinforcement Learning approach to learn an efficient policy that dynamically decides when measurements are to be processed at each sensor. Effectiveness of our proposed approach is validated through a numerical simulation with case study on smart sensing motivated by the Internet of Drones.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. H. Sun, Q. Guo, J. Qi, V. Ajjarapu, R. Bravo, J. Chow, Z. Li, R. Moghe, E. Nasr-Azadani, U. Tamrakar, G. N. Taranto, R. Tonkoski, G. Valverde, Q. Wu, and G. Yang, “Review of challenges and research opportunities for voltage control in smart grids,” IEEE Trans. Power Syst., vol. 34, no. 4, pp. 2790–2801, 2019.
  2. H. Erdem and V. Gungor, “On the lifetime analysis of energy harvesting sensor nodes in smart grid environments,” Ad Hoc Networks, vol. 75-76, pp. 98–105, 2018.
  3. P. K. Reddy Maddikunta, S. Hakak, M. Alazab, S. Bhattacharya, T. R. Gadekallu, W. Z. Khan, and Q.-V. Pham, “Unmanned aerial vehicles in smart agriculture: Applications, requirements, and challenges,” IEEE Sensors J., vol. 21, no. 16, pp. 17 608–17 619, 2021.
  4. K. Haseeb, I. Ud Din, A. Almogren, and N. Islam, “An energy efficient and secure iot-based wsn framework: An application to smart agriculture,” Sensors, vol. 20, no. 7, 2020.
  5. A. Bueno, M. Godinho Filho, and A. G. Frank, “Smart production planning and control in the industry 4.0 context: A systematic literature review,” Comput. Ind. Eng., vol. 149, p. 106774, 2020.
  6. D. Ivanov, S. Sethi, A. Dolgui, and B. Sokolov, “A survey on control theory applications to operational systems, supply chain management, and industry 4.0,” Annu. Rev. Control, vol. 46, pp. 134–147, 2018.
  7. M. Krugh and L. Mears, “A complementary cyber-human systems framework for industry 4.0 cyber-physical systems,” Manuf. Lett., vol. 15, pp. 89–92, 2018, industry 4.0 and Smart Manufacturing.
  8. B. Taner, R. Bhusal, and K. Subbarao, “A nested robust controller design for interconnected vehicles,” in AIAA Scitech Forum, 2020.
  9. V. Giammarino, S. Baldi, P. Frasca, and M. L. D. Monache, “Traffic flow on a ring with a single autonomous vehicle: An interconnected stability perspective,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 8, pp. 4998–5008, 2021.
  10. P. A. Rad, D. Hofmann, S. A. Pertuz Mendez, and D. Goehringer, “Optimized deep learning object recognition for drones using embedded gpu,” in Proc. IEEE ETFA, 2021, pp. 1–7.
  11. S. Laso, M. Linaje, J. Garcia-Alonso, J. M. Murillo, and J. Berrocal, “Deployment of apis on android mobile devices and microcontrollers,” in Proc. IEEE PerCom Workshops, 2020, pp. 1–3.
  12. S. Li, L. Da Xu, and S. Zhao, “5G internet of things: A survey,” J. Ind. Inf. Integr., vol. 10, pp. 1–9, 2018.
  13. N. Dal Fabbro, M. Rossi, G. Pillonetto, L. Schenato, and G. Piro, “Model-free radio map estimation in massive mimo systems via semi-parametric gaussian regression,” IEEE Wireless Commun. Lett., vol. 11, no. 3, pp. 473–477, 2022.
  14. W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE Internet Things J., vol. 3, no. 5, pp. 637–646, 2016.
  15. S. Yi, C. Li, and Q. Li, “A survey of fog computing: concepts, applications and issues,” in Proc. ACM Workshop Mobile Big Data, 2015, pp. 37–42.
  16. S. Chen, H. Wen, J. Wu, W. Lei, W. Hou, W. Liu, A. Xu, and Y. Jiang, “Internet of things based smart grids supported by intelligent edge computing,” IEEE Access, vol. 7, pp. 74 089–74 102, 2019.
  17. M. O’Grady, D. Langton, and G. O’Hare, “Edge computing: A tractable model for smart agriculture?” Artificial Intelligence Agriculture, vol. 3, pp. 42–51, 2019.
  18. W. Fang, Y. Zhang, B. Yu, and S. Liu, “Fpga-based orb feature extraction for real-time visual slam,” in Proc. ICFPT, 2017, pp. 275–278.
  19. Y. Chang, Y. Tian, J. P. How, and L. Carlone, “Kimera-multi: a system for distributed multi-robot metric-semantic simultaneous localization and mapping,” in Proc. IEEE ICRA, 2021, pp. 11 210–11 218.
  20. S. Li, Z. Zhang, R. Mao, J. Xiao, L. Chang, and J. Zhou, “A fast and energy-efficient snn processor with adaptive clock/event-driven computation scheme and online learning,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 68, no. 4, pp. 1543–1552, 2021.
  21. S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K. Chan, “When edge meets learning: Adaptive control for resource-constrained distributed machine learning,” in Proc. IEEE INFOCOM, 2018, pp. 63–71.
  22. L. Schenato, B. Sinopoli, M. Franceschetti, K. Poolla, and S. S. Sastry, “Foundations of control and estimation over lossy networks,” Proc. IEEE, vol. 95, no. 1, pp. 163–187, Jan 2007.
  23. A. Chiuso, N. Laurenti, L. Schenato, and A. Zanella, “Lqg cheap control subject to packet loss and snr limitations,” in Proc. ECC, July 2013, pp. 2374–2379.
  24. F. Li, M. C. de Oliveira, and R. Skelton, “Integrating information architecture and control or estimation design,” SICE JCMSI, vol. 1, pp. 120–128, 01 2011.
  25. E. Clark, S. L. Brunton, and J. N. Kutz, “Multi-fidelity sensor selection: Greedy algorithms to place cheap and expensive sensors with cost constraints,” IEEE Sensors J., vol. 21, no. 1, pp. 600–611, 2021.
  26. T. Devos, M. Kirchner, J. Croes, W. Desmet, and F. Naets, “Sensor selection and state estimation for unobservable and non-linear system models,” Sensors, vol. 21, no. 22, 2021.
  27. M. Alonso, H. Amaris, D. Alcala, and D. M. Florez R., “Smart sensors for smart grid reliability,” Sensors, vol. 20, no. 8, 2020.
  28. V. Tzoumas, L. Carlone, G. J. Pappas, and A. Jadbabaie, “Lqg control and sensing co-design,” IEEE Trans. Autom. Control, vol. 66, no. 4, pp. 1468–1483, 2021.
  29. G. Zardini, A. Censi, and E. Frazzoli, “Co-design of autonomous systems: From hardware selection to control synthesis,” in Proc. ECC, 2021, pp. 682–689.
  30. V. Tripathi, L. Ballotta, L. Carlone, and E. Modiano, “Computation and communication co-design for real-time monitoring and control in multi-agent systems,” in Proc. WiOpt, 2021, pp. 1–8.
  31. S. Chinchali, A. Sharma, J. Harrison, A. Elhafsi, D. Kang, E. Pergament, E. Cidon, S. Katti, and M. Pavone, “Network offloading policies for cloud robotics: a learning-based approach,” Auton. Robots, vol. 45, no. 7, pp. 997–1012, 2021.
  32. T. Taami, S. Krug, and M. O’Nils, “Experimental characterization of latency in distributed iot systems with cloud fog offloading,” in Proc. IEEE WFCS, May 2019, pp. 1–4.
  33. J. Rüth, R. Glebke, T. Ulmen, and K. Wehrle, “Demo abstract: Towards in-network processing for low-latency industrial control,” in Proc. IEEE INFOCOM WKSHPS, April 2018, pp. 1–2.
  34. L. Ballotta, L. Schenato, and L. Carlone, “Computation-communication trade-offs and sensor selection in real-time estimation for processing networks,” IEEE Trans. Netw. Sci. Eng., vol. 7, no. 4, pp. 2952–2965, 2020.
  35. Z. Shi, W. Yao, Z. Li, L. Zeng, Y. Zhao, R. Zhang, Y. Tang, and J. Wen, “Artificial intelligence techniques for stability analysis and control in smart grids: Methodologies, applications, challenges and future directions,” Applied Energy, vol. 278, p. 115733, 2020.
  36. S. Zilberstein, “Using anytime algorithms in intelligent systems,” AI Magazine, vol. 17, no. 3, p. 73, Mar. 1996.
  37. R. Tou and J. Zhang, “Imm approach to state estimation for systems with delayed measurements,” IET Signal Process., vol. 10, no. 7, pp. 752–757, 2016.
  38. A. Gopalakrishnan, N. S. Kaisare, and S. Narasimhan, “Incorporating delayed and infrequent measurements in extended kalman filter based nonlinear state estimation,” J. Process Control, vol. 21, no. 1, pp. 119–129, 2011.
  39. L. Greco, D. Fontanelli, and A. Bicchi, “Design and stability analysis for anytime control via stochastic scheduling,” IEEE Trans. Autom. Control, vol. 56, no. 3, pp. 571–585, 2011.
  40. T. Fedullo, A. Morato, F. Tramarin, L. Rovati, and S. Vitturi, “A comprehensive review on time sensitive networks with a special focus on its applicability to industrial smart and distributed measurement systems,” Sensors, vol. 22, no. 4, 2022.
  41. W. Yu, M. Jia, C. Liu, and Z. Ma, “Task preemption based on petri nets,” IEEE Access, vol. 8, pp. 11 512–11 519, 2020.
  42. C. J. C. H. Watkins and P. Dayan, “Q-learning,” Mach. Learn., vol. 8, no. 3, pp. 279–292, May 1992.
  43. R. S. Sutton, “Learning to predict by the methods of temporal differences,” Mach. Learn., vol. 3, no. 1, p. 9–44, aug 1988.
  44. T. Jaakkola, M. Jordan, and S. Singh, “Convergence of stochastic iterative dynamic programming algorithms,” in Adv. Neural Inf. Process. Syst., J. Cowan, G. Tesauro, and J. Alspector, Eds., vol. 6.   Morgan-Kaufmann, 1993.
  45. A. Abdessameud and A. Tayebi, “On consensus algorithms for double-integrator dynamics without velocity measurements and with input constraints,” Syst. Control. Lett., vol. 59, no. 12, pp. 812–821, 2010.
  46. A. Allan. (2019, Aug.) The big benchmarking roundup. https://aallan.medium.com/the-big-benchmarking-roundup-a561fbfe8719.
  47. S. Hossain and D.-j. Lee, “Deep learning-based real-time multiple-object detection and tracking from aerial imagery via a flying robot with gpu-based embedded devices,” Sensors, vol. 19, no. 15, 2019.
Citations (3)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up