2000 character limit reached
One Transformer for All Time Series: Representing and Training with Time-Dependent Heterogeneous Tabular Data
Published 13 Feb 2023 in cs.LG and cs.AI | (2302.06375v4)
Abstract: There is a recent growing interest in applying Deep Learning techniques to tabular data, in order to replicate the success of other Artificial Intelligence areas in this structured domain. Specifically interesting is the case in which tabular data have a time dependence, such as, for instance financial transactions. However, the heterogeneity of the tabular values, in which categorical elements are mixed with numerical items, makes this adaptation difficult. In this paper we propose a Transformer architecture to represent heterogeneous time-dependent tabular data, in which numerical features are represented using a set of frequency functions and the whole network is uniformly trained with a unique loss function.
- Benjelloun, O., Chen, S., Noy, N.: Google dataset search by the numbers. In: International Semantic Web Conference, Springer (2020) Borisov et al. [2023] Borisov, V., Broelemann, K., Kasneci, E., Kasneci, G.: DeepTLF: robust deep neural networks for heterogeneous tabular data. Int. J. Data Sci. Anal. 16(1), 85–100 (2023) Chen and Guestrin [2016] Chen, T., Guestrin, C.: XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Broelemann, K., Kasneci, E., Kasneci, G.: DeepTLF: robust deep neural networks for heterogeneous tabular data. Int. J. Data Sci. Anal. 16(1), 85–100 (2023) Chen and Guestrin [2016] Chen, T., Guestrin, C.: XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Chen, T., Guestrin, C.: XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Borisov, V., Broelemann, K., Kasneci, E., Kasneci, G.: DeepTLF: robust deep neural networks for heterogeneous tabular data. Int. J. Data Sci. Anal. 16(1), 85–100 (2023) Chen and Guestrin [2016] Chen, T., Guestrin, C.: XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Chen, T., Guestrin, C.: XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Chen, T., Guestrin, C.: XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017) Padhi et al. [2021] Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P.L., Ross, J., Nair, R., Altman, E.: Tabular transformers for modeling multivariate time series. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2021) Arnab et al. [2021] Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV (2021) Mildenhall et al. [2020] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) Rahaman et al. [2019] Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C.: On the spectral bias of neural networks. In: ICML (2019) Devlin et al. [2019] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, (2019) Bao et al. [2022] Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Bao, H., Dong, L., Wei, F.: BEiT: BERT pre-training of image transformers. ICLR (2022) Ramesh et al. [2021] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021) Szegedy et al. [2016] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (2016) Lyu et al. [2022] Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Lyu, F., Tang, X., Zhu, H., Guo, H., Zhang, Y., Tang, R., Liu, X.: OptEmbed: learning optimal embedding table for click-through rate prediction. In: Hasan, M.A., Xiong, L. (eds.) Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022) Shankaranarayana and Runje [2021] Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Shankaranarayana, S.M., Runje, D.: Attention augmented convolutional transformer for tabular time-series. In: 2021 International Conference on Data Mining, ICDM 2021 - Workshops (2021) Huang et al. [2020] Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.S.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678 (2020) Schäfl et al. [2022] Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Schäfl, B., Gruber, L., Bitto-Nemling, A., Hochreiter, S.: Hopular: Modern hopfield networks for tabular data. arXiv:2206.00664 (2022) Wu et al. [2022] Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Wu, Y., Rabe, M.N., Hutchins, D., Szegedy, C.: Memorizing transformers. In: ICLR (2022) Kossen et al. [2021] Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Kossen, J., Band, N., Lyle, C., Gomez, A.N., Rainforth, T., Gal, Y.: Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. In: NeurIPS (2021) Gorishniy et al. [2021] Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. In: NeurIPS (2021) Somepalli et al. [2021] Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: SAINT: improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342 (2021) Han et al. [2022] Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Han, H., Xu, J., Zhou, M., Shao, Y., Han, S., Zhang, D.: LUNA: language understanding with number augmentations on transformers via number plugins and pre-training. arXiv:2212.02691 (2022) Solatorio and Dupriez [2023] Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Solatorio, A.V., Dupriez, O.: REaLTabFormer: Generating realistic relational and tabular data using transformers. arXiv:2302.02041 (2023) Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N.D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M.S., Krishna, R., Kuditipudi, R., al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021) Narayan et al. [2022] Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Narayan, A., Chami, I., Orr, L.J., Ré, C.: Can foundation models wrangle your data? Proc. VLDB Endow. 16(4), 738–746 (2022) Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. arXiv:2005.14165 (2020) Borisov et al. [2022] Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., Kasneci, G.: Language models are realistic tabular data generators. arXiv:2210.06280 (2022) Jiang et al. [2023] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: StructGPT: A general framework for large language model to reason on structured data. arXiv:2305.09645 (2023) Wen et al. [2022] Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., Sun, L.: Transformers in time series: A survey. arXiv:2202.07125 (2022) Berka [1999] Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Berka, P.: Workshop notes on Discovery Challenge PKDD’99. (1999). https://sorry.vse.cz/~berka/challenge/pkdd1999/berka.htm Prokhorenkova et al. [2018] Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. In: NeurIPS (2018) Christ et al. [2018] Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package). Neurocomputing 307, 72–77 (2018) Xu [2020] Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020) Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
- Xu, Z.: Loan Default Prediction with Berka Dataset. https://towardsdatascience.com/loan-default-prediction-an-end-to-end-ml-project-with-real-bank-data-part-1-1405f7aecb9e (2020)
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.