Counter-Empirical Attacking based on Adversarial Reinforcement Learning for Time-Relevant Scoring System
Abstract: Scoring systems are commonly seen for platforms in the era of big data. From credit scoring systems in financial services to membership scores in E-commerce shopping platforms, platform managers use such systems to guide users towards the encouraged activity pattern, and manage resources more effectively and more efficiently thereby. To establish such scoring systems, several "empirical criteria" are firstly determined, followed by dedicated top-down design for each factor of the score, which usually requires enormous effort to adjust and tune the scoring function in the new application scenario. What's worse, many fresh projects usually have no ground-truth or any experience to evaluate a reasonable scoring system, making the designing even harder. To reduce the effort of manual adjustment of the scoring function in every new scoring system, we innovatively study the scoring system from the preset empirical criteria without any ground truth, and propose a novel framework to improve the system from scratch. In this paper, we propose a "counter-empirical attacking" mechanism that can generate "attacking" behavior traces and try to break the empirical rules of the scoring system. Then an adversarial "enhancer" is applied to evaluate the scoring system and find the improvement strategy. By training the adversarial learning problem, a proper scoring function can be learned to be robust to the attacking activity traces that are trying to violate the empirical criteria. Extensive experiments have been conducted on two scoring systems including a shared computing resource platform and a financial credit system. The experimental results have validated the effectiveness of our proposed framework.
- W. Wang, C. Lesner, A. Ran, M. Rukonic, J. Xue, and E. Shiu, “Using small business banking data for explainable credit risk scoring,” AAAI, vol. 34, no. 08, pp. 13 396–13 401, 2020.
- S. Teerasoponpong and A. Sopadang, “Decision support system for adaptive sourcing and inventory management in small-and medium-sized enterprises,” Robotics and Computer-Integrated Manufacturing, vol. 73, p. 102226, 2022.
- S. Israel, A. Caspi, D. W. Belsky, H. Harrington, S. Hogan, R. Houts, S. Ramrakha, S. Sanders, R. Poulton, and T. E. Moffitt, “Credit scores, cardiovascular disease risk, and human capital,” PNAS, vol. 111, no. 48, pp. 17 087–17 092, 2014.
- G. Guo, F. Zhu, E. Chen, L. Wu, Q. Liu, Y. Liu, and M. Qiu, “Personal credit profiling via latent user behavior dimensions on social media,” in PAKDD. Springer, 2016, pp. 130–142.
- S. Maldonado and G. Paredes, “A semi-supervised approach for reject inference in credit scoring using svms,” in ICDM. Springer, 2010, pp. 558–571.
- F.-C. Li, “The hybrid credit scoring strategies based on knn classifier,” in Sixth International Conference on Fuzzy Systems and Knowledge Discovery, vol. 1. IEEE, 2009, pp. 330–334.
- E. Dumitrescu, S. Hue, C. Hurlin, and S. Tokpavi, “Machine learning for credit scoring: Improving logistic regression with non-linear decision-tree effects,” European Journal of Operational Research, vol. 297, no. 3, pp. 1178–1192, 2022.
- N. Kozodoi, J. Jacob, and S. Lessmann, “Fairness in credit scoring: Assessment, implementation and profit implications,” European Journal of Operational Research, vol. 297, no. 3, pp. 1083–1094, 2022.
- S. Bhatnagar, M. Ghavamzadeh, M. Lee, and R. S. Sutton, “Incremental natural actor-critic algorithms,” Advances in neural information processing systems, vol. 20, pp. 105–112, 2007.
- I. Grondman, L. Busoniu, G. A. Lopes, and R. Babuska, “A survey of actor-critic reinforcement learning: Standard and natural policy gradients,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 1291–1307, 2012.
- J. Schulman, P. Moritz, S. Levine, M. I. Jordan, and P. Abbeel, “High-dimensional continuous control using generalized advantage estimation,” in ICLR, 2016.
- J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
- M. Cuturi and M. Blondel, “Soft-dtw: a differentiable loss function for time-series,” in ICML. PMLR, 2017, pp. 894–903.
- M. L. littman, “Markov games as a framework for multi-agent reinforcement learning,” in ICML, ser. ICML’94. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1994, p. 157–163.
- J. Pérolat, B. Scherrer, B. Piot, and O. Pietquin, “Approximate dynamic programming for two-player zero-sum markov games,” in Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, ser. JMLR Workshop and Conference Proceedings, F. R. Bach and D. M. Blei, Eds., vol. 37. JMLR.org, 2015, pp. 1321–1329.
- J. Xiong, Q. Wang, Z. Yang, P. Sun, L. Han, Y. Zheng, H. Fu, T. Zhang, J. Liu, and H. Liu, “Parametrized deep q-networks learning: Reinforcement learning with discrete-continuous hybrid action space,” arXiv preprint arXiv:1810.06394, 2018.
- L. Glover, “Creditworthy: A history of consumer surveillance and financial identity in america by josh lauer,” Journal of Intellectual Freedom & Privacy, vol. 2, no. 3-4, pp. 33–34, 2018.
- Y. Zhang, M. A. Orgun, R. A. Baxter, and W. Lin, “An application of element oriented analysis based credit scoring,” in ICDM, P. Perner, Ed., vol. 6171. Springer, 2010, pp. 544–557.
- Z. Nana, W. Xiujian, and Z. Zhongqiu, “Game theory analysis on credit risk assessment in e-commerce,” Information Processing & Management, vol. 59, no. 1, p. 102763, 2022.
- K. Lei, Y. Xie, S. Zhong, J. Dai, M. Yang, and Y. Shen, “Generative adversarial fusion network for class imbalance credit scoring,” Neural Computing and Applications, vol. 32, no. 12, pp. 8451–8462, 2020.
- G. Ghalme, S. Gujar, A. Kumar, S. Jain, and Y. Narahari, “Design of coalition resistant credit score functions for online discussion forums,” in AAMAS, 2018, pp. 95–103.
- S. Pan and T. Ding, “Social media-based user embedding: A literature review,” in IJCAI, 2019.
- B. Liu, X. Sun, Z. Ni, J. Cao, J. Luo, B. Liu, and X. Fu, “Co-detection of crowdturfing microblogs and spammers in online social networks,” World Wide Web, vol. 23, no. 1, pp. 573–607, oct 2019.
- Y. Wu, D. Lian, Y. Xu, L. Wu, and E. Chen, “Graph convolutional networks with markov random field reasoning for social spammer detection,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 01, pp. 1054–1061, apr 2020.
- Y. Dou, G. Ma, P. S. Yu, and S. Xie, “Robust spammer detection by nash reinforcement learning,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, aug 2020.
- X. Sun, B. Liu, L. Ai, D. Liu, Q. Meng, and J. Cao, “In your eyes: Modality disentangling for personality analysis in short video,” IEEE Transactions on Computational Social Systems, pp. 1–12, 2022.
- S. Zhao, G. Ding, J. Han, and Y. Gao, “Personality-aware personalized emotion recognition from physiological signals,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, jul 2018.
- X. Sun, H. Cheng, B. Liu, J. Li, H. Chen, G. Xu, and H. Yin, “Self-supervised hypergraph representation learning for sociological analysis,” IEEE Transactions on Knowledge and Data Engineering, pp. 1–12, 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.